The Technical Side Of Balancing/Testing

Gods And Generals, Mort Kunstler

I don't have anything more to talk about on the game itself this morning. Since I was pulling my hair out all-day yesterday figuring out how to use the advanced features of Microsoft Excel properly. But I might as well talk about what I'm doing, which is the technical setup of a model/unit database and a semi-automated testing system.

So far, when playing around with odds and whatnot, I've been working in Excel with everything on a single sheet. Unit/Weapon stats, comparison charts, etc.

Obviously, this isn't going to cut it going forward. We simply going to have too many units and potential interactions to juggle, and the one-sheet process is far too manual. If we try and balance each and every unit this way, we'll drown ourselves in a sea of minutiae and not get anywhere.

So...

What I'm working on now is a multi-sheet approach where the data is broken up into different lists that are linked together automatically. You start with some defined model classes, add weapons and armor in another list to make a troop type, and then you construct units as collections of different troop types. Another page will then let you select which defined unit you want and construct a stat listing automatically. While other pages will let us pick 2 defined units and see how they compare to each other in some predetermined test scenarios.

Eventually, we'll be able to tweak a stat in one list and have that change propagate itself on through to the unit listings and tests. Once we have a unit cost mechanic in place, that will be updated too.

Why Do It This Way?

In a word... Speed. With a proper system like this in place we can not only crank out new units and races fast, but we can also test much faster as well.

How does this decrease our testing time? Well... Certain things, like shooting odds, are inherently mathematical. We don't need to play 20+ games to determine the ratio of hits or average wounds from one troop type to another. Especially since even 100 games wouldn't be a large enough sample size to produce a meaningful average. It's also ridiculous to think that we'll want to take 4 hours to play a 1 hour game so that every little detail of the engagement can be recorded.

So we automate the straight odds stuff and concentrate on quantifying the inherently fuzzy stuff. Such as:

1. "Is This Mechanic Fun?"
2. "Is Theory X Validated?"
3. "Is Having The First Turn Too Important? How Can We Address That?" 
4. "How Important To A Unit's Proper Costing Is Shooting Ability Compared To Assault Ability?"

For the quantifiable fuzzy-stuff, like #4, we work balancing weights into our top-level cost equations. We'll know the raw numbers for shooting and assault effectiveness from the math. We then weight those numbers according to our testing. So as we go along, if we make a change that causes Assault to be more important than it was, we can tweak the high level weights and have the costs for all models update automatically.

It's a technical approach that I think will give us a huge edge moving forward.

While I'm pretty sure that Battlefront has a similar system/approach (given the speed at which they kick out new books that are amazingly well balanced/costed), it's also pretty obvious that GW doesn't. If they had such a system, you wouldn't see new models appear, like the Pyrovore, that are way-overcosted for their role. That tells me that they cost things intuitively, and according to limited playtesting instead of through any sort of rigorous technical system.

The other big advantage is that it forces us to understand the system we're creating at a very low level. Because every change to a mechanic will have to be propagated through the unit system quantifiably.

Thoughts?