Test It Real Good

Bucking Bronco, Frank Wootton

CaulynDarr here. I thought I'd take a break from contemplating the finer points of alien bug reproduction to discus testing.

In my schedule post last week, I blocked out time for testing in between each development iteration. Given that the end of the first iteration is coming up, it would be a good time to talk about what needs to happen to make our testing efforts successful.

Good testing needs objectives and goals just like development work.  Not every phase of testing will be the same.  We'll be looking at different things and trying to stress the system in different ways.   The things we find in testing will feed into the specific type of work we do in the next iteration.

If you remember from last week, Draft 1 was to lay down the basic rules for movement, shooting, assaulting, and terrain.  Since this leaves much of the game still undefined(including pretty much all of the units) the type of testing we can do at this point is fairly limited.  So you may be wondering why I gave it a whole week? Well, for one reason, every project ends up having a minimum amount of time you can give to a task based on a whole slew of reason.  I figured with our distributed locations and busy real lives, it would be hard to get anything significant done in less than a week.   The second reason is that our testing cycles will also function as a peer review of the material produced so far.

M42 Alpha Development Testing Plan

Draft 1 Testing

Draft 1 testing is primarily an internal testing step.  This doesn't mean that we won't let you guys see the rules or want to know what you think(We'll probably open up a thread for you guys to give us your opinions).  What it means is that the core project team has the bandwidth to discuss testing results much more rapidly and in-depth.  We want to get our core team all on the same page with a good understanding of the basic rules.  We are also looking for a birds-eye view or possible rules issues and interactions.  We aren't necessarily going to try and break the rules...yet.

• Objective 1 - The first objective of draft 1 testing is to proof-read the draft rules.  Part of this will be checking for the basic grammar and spelling stuff.  More importantly we will be checking that the wording of the rules are clear and concise.  Every rule should be vetted to make sure that it cannot be described in simpler terms.
• Objective 2 - Secondly, we want to sanity check the rules mechanics.  Here we want to make sure that all the rules we have written translate to the table top.  We need to find rules that are awkward to follow, or so unimportant as to be forgotten in play even by the people who wrote them(trust me this can happen).  We also need to keep an eye on deficiencies; places where it would be handy to add a rule or two to clarify things.

I expect about a 50/50 spit in effort between points 1 and 2.  Based on our results we may revise the content of Draft 2. 

Draft 2 Testing 

Draft 2 testing is pretty much a repeat of draft 1, but here we should be spending 75% of our time focusing on objective 2.

Alpha Testing

Here's the first really meaty testing we'll be doing.  Internally we'll probably do some more proof-reading, but we'll need more community testing to really achieve our objectives.  Here we want people to play the game and tell us what they think.  People should drill down into the mechanics and find out where the more crunchy problems in the game are.  People should still be fairly gentle with the system.  The real gloves-off-break-it-apart-testing will come as part of the beta fine tuning.

The core team should also have a good idea of the strengths and weaknesses of the system by this time.  Because of that, we should be able to provide test case scenarios and questionaries to help focus people's testing efforts.  

• Objective 1 - Find out if the game is fun to play.
• Objective 2 - Find out if the rules are clear and can be easily explained to non-project members.
• Objective 3 - Find out if the game has the potential tactical depth to keep people interested.

Those are big questions.  If you guys aren't satisfied with the answers, we're going to have to go back and repeat the Alpha again and again until you do.  Otherwise, if you guys love it, then it will be time to begin beta development.