Stencyling Around

In a recent interview, Sandy Duncan, CEO of YoYo Games, was asked about his opinion of upcoming competitors. While he didn’t name us directly, he said that none of the ones he’s seen is “adequately staffed or funded to pull it off.” Clearly, he’s referring to us and a few others, though technically, we’re not open source, nor are we a project in the sense that this is a hobby.

Sandy is quite right about the staffing and the funding. We are a self-funded venture, which means that everything is paid out of our pockets. We’ve put in a low 5 figure amount right now, which squarely puts us in the real business category (as opposed to a project) but is still a very small amount compared to the $2 million that YYG has, or the $10 million that Kongregate has gotten. Because we’re self-funded, we’re unable to hire employees or to work on this full time. This is why it’s taken 10 months to get to a near-beta state. To our credit, we know peripherally related competitors, in beta, that have taken 1 or 2 years working full time to reach that state (with paid employees), so we’re doing awfully well for part-time workers on such an ambitious effort. So while we boast having fixed 130 bugs last month on top of substantial feature work, the reality is that if you aren’t at least that productive, Sandy is right, you’re a non-starter. You literally need to be as productive as several full-time employees to even be competitive

In the world of startups, money and staffing aren’t the only considerations for success. They certainly make it easier, but it comes down to having the right vision, execution, business plan, and a good dose of luck. If you have a lot  of happy users (read: at least 100,000) and can do that while making money, you are a success (business wise at least), no matter what others say.

Sandy is definitely right about one thing - it takes an immense amount of time, dedication and knowhow to make a good game creation platform. That’s why most quit midway through, restart many times over or finish and end up with something that nobody cares to use because it has poor usability. Or sometimes, it has great usability but just wasn’t solving the right problem and is now a whole bunch of work spent for naught. You know, like all those niche creators where everything you create feels the same?

I can feel why Sandy is dismissive of his competition - most competitors I’ve seen are non-starters. Nobody has really come up with a viable alternative to Game Maker (besides Torque, but that costs $100 and is irrelevant to us on cost alone). We’re aware of several other startups too, but of the ones that are out in the wild, we still see nobody who’s offering something that’s any better than Game Maker. There are occasional fragments of really good ideas, but the overall execution isn’t there, and this isn’t my opinion - the traffic numbers show that these startups aren’t gaining traction.

In the end, while competition is an inevitable part of any business, let’s remind ourselves that we’re not out to “beat” Game Maker. We’re here to execute on our vision and make the best product for our target audience. That’s how a successful product is made, and if we’re to be competitive, it’s by doing that. It’s not done by staring at the competition, copying features or engaging in a war of words. I surely hope that this is the last time I have to do that!

Both taken with my 17-55 at f/2.8 and ISO 1600.

Gotcha with the title. But honestly, bugs are good in many ways that aren’t obvious at first glance. This article will talk about how we approach bugs at Stencyl and how this part of experience has panned out for us.

Bugs != Buggy Product

The first lesson about bugs is that for a decent project, there’s not a whole lot of correlation between bug count and how buggy that piece of software is. A high bug count is indicative of how much feedback that software has gotten. Feedback is good because it opens a conversation between you and the bug reporters, whether they are your users or technical peers. Fixing bugs successfully and collaborating with these people will help build mutual respect, and that’s something that’s incredibly difficult to build without a proper process in place.

How I Approach Bugs

I approach bugs as if they were puzzles. Within 10 seconds, I bucket them into one of three categories.

1. Can fix in 5 minutes.
2. Can fix, but it’ll take some time.
3. Don’t know what the heck is going on or need more information.

All of the (1) bugs are immediately dispensed of on the spot, regardless of priority. My simple reasoning is this - I don’t like trivial bugs piling up, and an excessive amount of trivial bugs can mask the more important ones and just make the whole queue look a lot bigger than it is.

The bugs in group (2) are handled on a case to case basis. If I can fix them rather quickly, I might tackle them at the end of the night or in the morning in my spare time. Otherwise, I prioritize, group and schedule them.

Bugs in (3) are the most fun and grueling to fix. Sometimes, I’ll need more information from the reporting user. Sometimes, I just have to set it aside and let the ideas flow in in the shower or car. It really isn’t something you can to brute force - it’ll come to you in time.

Discovering the Root Cause

After accepting a bug and even while I bucket my bugs, I begin formulating a hypotheses about the root cause(s) of the bug. Sometimes, it comes to me right away, and I know exactly what part of the code to look in and what is causing it, a non-trivial task now that the codebase is well above 100,000 lines and 600 files of source code.

If the cause is non-obvious, unlike some others, I don’t reach for my debugger or print statements right away (and to be honest, I rarely use a debugger). Nor do I look at the output or stack traces. I start at the fundamental level and figure out what IS possibly going wrong. This means that I have to formulate a mental model of the process, play out some scenarios in my head and zone in on the code that’s responsible. This way, I don’t rely on tools as a crutch. I let my mind exercise itself and by this method, know my code that much better than if I had jumped right away for the tools. More often than not, I am able to solve a problem this way.

If I still cannot grasp the problem, then I start bringing in the big guns. If it’s a crash, a stack trace is usually good enough. If it’s a logical flaw, I drill down on the problem, using tools to assist. I employ Toyota’s 5 Why’s Method to discover the cause, again to take an analytical approach to the problem.

All in all, bug fixing should be viewed as a critical thinking exercise where you reinforce your knowledge of the code and occasionally learn something new. It should not be viewed as grunt work, and if you diligently approach this in a constructive way, you’ll find yourself fixing bugs quicker, introducing fewer bugs and understanding your code much more. That’s how I was able to pull off a 90 -fixed -bugs boup in 3 weeks on top of several features I was developing.

We have a QA Process

Back in March, I proposed starting a QA (quality assurance) program and appointed Fox, one of our newer testers at the time, to the task, which was to proactively file bugs and verify my fixes for bugs. At the time, it might have sounded like I’d jumped the gun, but in retrospect, it was one of the best decisions I could have made for Stencyl.

For one thing, having a formal QA process in place allowed such a program to organically develop and refine itself over time. It’s better to start things earlier rather than force them in later.

Another benefit is that it developed an engineering-centric subculture in which our more technically oriented testers were bringing their own knowledge into the fray and engaging in meaningful discussions whether it involved asking be questions about the system or proposing their own solutions to problems. Those kinds of discussions reach far beyond the bug fixing realm and contribute to the larger success of the venture

Getting back to the original process, we started with a forum-based approach and had 3 forums: Bugs, Fixed Bugs and Closed Bugs. Like a Bugzilla-ish system, people would file a bug in Bugs.  I would evaluate and fix the bug and move it to Fixed Bugs. The QA contact would then verify the bug using a battery of tests that we joint developed and would finally move it to closed if fixed or back to Bugs if it needed further work. It was a simple model, but it worked for many months until we started expanding.

Today, we have more categories, an additional QA Contact (Greg) and many more issues to track. Since March, 700 issues have been filed and the volume (both fixing and reporting new issues) has picked up substantially. All in all, I am elated with the current status of our QA program, and it’ll only refine further as the software matures and the website part of our venture comes into play.

Conclusion

This QA process stemmed from my industry experience, and it’s really something you can’t come up with on your own if you work on a project with other developers who have yet to work in a real company. While working in a bigger company has its ups and its downs, the power of process, when applied properly, is a real boon, and this is one example of that.

While users will be happy to know that we place a strong emphasis on quality, the real win here is in the intangible stuff. It’s the valuable feedback, the discussions we’ve held and the general environment that count for a lot, and in a way, this culture is just as pivotal to our success as the technology itself. I’ve realized that there’s a whole lot more to software than just developing it - it’s the singular, unified vision you have, the decisions the team makes and all that expert, domain-specific knowledge that count.

- Jon

It’s been quite a while since my last entry. A lot has been happening around here, progress and work-wise. I won’t attempt to go into detail about what’s been done since there’s been so much done. Let’s summarize with a few statistics instead.

Features that were implemented since July
Game Center, Updater, Publisher, the player (engine), Snippet Builder, Groups Editor, Collisions Editor, Controls Editor, Physics Editor (part of the Actor Creator), moving the player to Box2D, adding Joint Support (both in our API and our GUI) and a bevy of look and feel improvements.

Bugs Fixed since July
~250

Bugs/Issues Still Open
~250 (roughly a 90 / 160 split between bugs and RFE’s)

Bugs Fixed since November 1st
90 (it’s been a doozy of a month - that works out to nearly 5 bugs per day on top of feature work!)

Release Cycles since July
39

Languages Localized to (besides English and these are full actively updated localizations)
10

I’m very thankful that we have such a dedicated QA effort going on here. Fox, Greg and Alexin have been gracefully filing bugs and verifying my fixes to bugs. The volume has considerably risen to the point where we actually have a queue of bugs (2o at this moment) that remain to be verified, despite that queue being worked on every day.

There’s still quite a bit more work to complete, but at the current pace I’m moving, we’re still on track. On track for what? You will hear more about that in the next official update. For now, take heart in knowing that we’re working on this at full capacity and are as eager to see this in the hands of beta testers as you are.

- Jon

Hainan Chicken
42mm, f/2.8, 1/60s

55mm, f/2.8, 1/250s

Hamachi
60mm, f/5.6, 1/30s

The harsh shadow can’t really be helped since this was shot in a restaurant, like all others and filling with a flash is out of the question.

Spicy Salmon Roll
60mm, f/5.6, 1/30s

Grilled Ox Tongue
60mm, f/5.6, 1/50s

Making a Java app look “native” on Mac is a hard problem on many levels. I’m not talking about slapping the “Aqua” look and feel or setting the system property to make the menu bar go to the top. I’m talking about the 10 million “little things” that make a Mac app feel like one. Architecture-wise, you also end up with a mess of if-statements and refactor your code using an AbstractFactory or some other pattern to keep things clean.

I spent a good night tackling some of these “little things” and Googling around, and while each one is tiny, they add up to a much better experience, and I guarantee that you Mac users out there will find StencylWorks to be one the most Mac-friendly Java programs out there and easily the best feeling Mac game creator out there, barring Unity3D, which is a native Mac app.

For those of you who haven’t figured it out by now, I do have a MacBook Pro, which has let me do this and a lot more, so to all Mac users out there, you’re covered well and won’t feed any my users dog food like I’ve seen some others do just to rush the product out the door.