As I’ve worked on playtesting the “end run” strategy, I’ve realized that I didn’t set the test up quite correctly. Just playing ten games using the strategy and ten games without isn’t a reliable way to get good data, because the experiment has too many variables. If the end run wins all ten of its games and going up the middle invariably loses, is that because going up the middle is weak or because the arrangement of terrain and searchers happened to be more favorable in the end run games?

To get useful data I have to do two things. One is define what constitutes an end run and what is going up the middle. Leaving those terms loose invites uncertainty as to whether I’m testing what I mean to be.

The other is tracking a lot more data than I have been. To compare the strategies fairly, the possibility that one of them got easier games has to be excluded. That means making sure they’re competing in the same arena–identical placements of terrain, starting locations of searchers, and even searcher moves.

I’m now playing test games with all of that in mind. “Going up the middle” means that no player token can ever be moved closer to either side of the board than six spaces (so five columns are disallowed); an “end run” means that at least one of the player tokens moves into that forbidden space, and stays there until the game ends. In each game I track the terrain and the searchers so that I can play the exact same scenario out with both strategies.

Unfortunately, I’m going to have to push back the end of this new project: I’ve had to toss out a fair amount of work, and playing the game with all the information tracking takes a fair amount of time. However, I’m confident that the results will be a lot more useful because of this new approach. I’ll give a status update in two weeks.