Theory: Playtesters and Designers Are Not In a Hierarchical Relationship

I saw an interview a long time ago–I regret that I can’t find it now–in which a game designer was asked how much of a contribution a playtester has to make before he or she gets credited as a designer. Like everyone who’s been to law school I have an aversion to “fighting the hypothetical,” but that question has always struck me as coming from an inherently wrong place. It assumes that playtesters are junior designers, and they are no such thing. Playtesting and designing are separate skills deserving of separate respect.

Good designers are not necessarily good playtesters, and vice-versa. Design ability does not confer the patience to test a game over countless permutations, nor does it grant the unique insight that enables great playtesters to look at a game and see unexpected ways in which a player might approach it. What’s more, design ability does not include the communication skills that are so important to giving effective playtest feedback. On the flip side, playtesting experience may not prepare someone to confront a blank sheet of paper and find a game there. An adept and valuable playtester may also have insight that primarily goes in one direction: he or she may be able to tell that something isn’t working, without necessarily having an opinion as to how it might be fixed. Playtesters and designers simply bring different things to the table.

Trying to fit playtesters and designers on the same ladder undervalues both. It elides the many skills each possesses, collapsing them into “people who make games” in order to force similarities that aren’t necessarily there. They should each be understood as having their own hierarchy, and should be measured against those in the same field, so that they can be evaluated according to how well they do their respective jobs instead of how they do each others’.

Efforts to “promote” playtesters into designers grow, I think, out of a sense that playtesters’ contributions are undervalued and that making them into designers will get them the recognition they deserve. I appreciate where people with that goal are coming from, but turning skilled playtesters into designers doesn’t do anything to get playtesting respect. If anything, it suggests that playtesters don’t need to be respected until they’ve “graduated” to designer status. That cure may really be worse than the disease.

The right solution here is simply to push recognition for playtesters. Acknowledge them more prominently. When you meet a good one, recommend him or her to others. Don’t cast playtesters as merely junior designers. Recognize and value the unique skills they bring to the table.

The Case Study: A Different Searcher Setup?

I was thinking about Over the Next Dune’s terrain, and how it had once been placed entirely randomly but was now spread across the board. Then I thought, “why aren’t the searchers set up that way?”

The answer, I realized, was “because I never thought of it.”

In the first versions of OtND to have both terrain and searchers, both were distributed randomly. Under that system the terrain sometimes fell entirely on one side of the board, or otherwise created a channel that dictated how people moved. Decisions in those games were less interesting than games where the terrain didn’t make such clear suggestions, so the rules were changed to space the terrain out. That proved sufficient to resolve the channels problem. As a result, I never gave consideration to whether spacing out the searchers would also be desirable.

There are advantages to distributing the searchers across the colored starting areas in the same fashion as the terrain. It’s easy to teach, because it’s consistent–each area gets one piece of terrain and one searcher. It also involves fewer types of dice, which is at least convenient.

On the other hand, simplicity isn’t the only value. Intuitively it seems like the game should get more interesting when the searchers start out evenly spaced, since they’re more likely to spread over the board and make players deal with them. However, there’s no way to know without trying it.

I’m not going to make this change now, since testing for the “end run” is still ongoing. However, I’m filing it away for afterward. If you get a chance, feel free to try this change out and let me know what you discover.

The Case Study: Redesigning the Experiment

As I’ve worked on playtesting the “end run” strategy, I’ve realized that I didn’t set the test up quite correctly. Just playing ten games using the strategy and ten games without isn’t a reliable way to get good data, because the experiment has too many variables. If the end run wins all ten of its games and going up the middle invariably loses, is that because going up the middle is weak or because the arrangement of terrain and searchers happened to be more favorable in the end run games?

To get useful data I have to do two things. One is define what constitutes an end run and what is going up the middle. Leaving those terms loose invites uncertainty as to whether I’m testing what I mean to be.

The other is tracking a lot more data than I have been. To compare the strategies fairly, the possibility that one of them got easier games has to be excluded. That means making sure they’re competing in the same arena–identical placements of terrain, starting locations of searchers, and even searcher moves.

I’m now playing test games with all of that in mind. “Going up the middle” means that no player token can ever be moved closer to either side of the board than six spaces (so five columns are disallowed); an “end run” means that at least one of the player tokens moves into that forbidden space, and stays there until the game ends. In each game I track the terrain and the searchers so that I can play the exact same scenario out with both strategies.

Unfortunately, I’m going to have to push back the end of this new project: I’ve had to toss out a fair amount of work, and playing the game with all the information tracking takes a fair amount of time. However, I’m confident that the results will be a lot more useful because of this new approach. I’ll give a status update in two weeks.

The Case Study: Updated Print-and-Play File

As I’ve played Over the Next Dune I’ve found that the terrain pieces, while printer ink-friendly, have inconvenient overlap into neighboring spaces. That overlap can create confusion when multiple things are near each other; if a space has terrain on three sides, it can look very much like there’s terrain in the space as well.

To resolve that issue I reduced the border size on the terrain pieces and filled them in with a (hopefully) desert-esque color. It should be easier to tell at a glance where terrain is and where it isn’t.

I also tried to shore up terrain piece number 4’s structural integrity with cross-pieces that still leave the spaces underneath mostly visible.

No changes were made to any pages other than the terrain, so it isn’t necessary to reprint everything. If you come across any problems with the new pieces, let me know!

Over the Next Dune – Print and Play – 4-2-14

Theory: Themes and Expectations

In thinking about themes for Over the Next Dune, I was reminded of Fantasy Flight Games’ Android. Android was widely, and in my view unfairly, criticized for having a divide between its gameplay and its narrative. It’s a cautionary tale about how themes are interpreted: players rely on a theme’s common associations, and if you’re going to do something different you have to help players bridge the gap.

In Android each player is a private detective trying to solve a crime. At the start of the game there are several suspects. Players put tokens on the suspects which represent evidence that points toward or away from that individual being the perpetrator. At the end of the game whichever suspect the evidence weighs most heavily against is arrested.

Many players hated the way Android handles investigation. The game never tells players who “really” did it, and there’s no smoking gun. It doesn’t flow like a conventional police procedural, where the investigators follow a trail of evidence from A to B to C until arriving at an irrefutable conclusion. Rather, evidence simply accumulates until there’s enough to declare someone the responsible party. A lot of people felt that they were being asked to do a bad investigation, and that the whole game was basically an exercise in framing someone.

However, in my experience as an attorney the accumulation of evidence is exactly how most criminal investigations go. It’s unusual to “break the case open” and be 100% certain about who committed a crime. Even theoretically objective, science-driven evidence like DNA analysis is frequently subject to interpretation and vulnerable to error. Most cases involve evidence that’s still less reliable, like eyewitnesses (studies have shown that eyewitnesses are frequently wrong, even when they say they’re sure about what they saw) and peoples’ recounting of conversations (try to remember everything you said to someone this morning, word for word–it’s not easy, and that’s with no pressure!). All the police can usually do is gather these pieces of evidence, knowing that they aren’t what one might wish for, and see if they sum up to an adequate level of proof.

Android’s problem isn’t that its investigation was implemented badly. To the contrary, I would say that Android’s investigation is unusually realistic. The problem is that many players’ expectations aren’t met. A lot of people go in anticipating the type of procedural story we see on TV and read in books, with a clear right answer to find, and they’re inevitably disappointed.

The lesson of Android, I feel, is that in choosing a theme designers must keep in mind what players will expect. If a game is going to provide something different, the designer needs to go out of his or her way to help the players see where the game is coming from. Android neither delivered what players were looking for nor explained why it wasn’t going to, and so it has consistently been criticized for not doing things it was never meant to do.

I’ll wrap up with an I’m-not-getting-paid-for-this moment: Android can still be found on the shelves and at significant markdowns online. I encourage you to pick it up. It’s much different from other games on the market, and is worth trying out.

The Case Study: More Playtesting

I’ve done more playtesting with the players starting in the middle of the board, and have continued to be pleased with the results. However, I have to admit that so far the results are all anecdotal. “It feels like this is working” is useful so far as it goes, but part of the goal of this blog is to be able to be more concrete.

A few weeks ago there was a “playtesting project“–general playtests that highlighted some rules issues and the need to have more players involved in rescues. I think it’s time for another project, this time focused on the “end run.” I’m planning to play ten games over the next two weeks in which I force the end run no matter the situation, and ten games in which I charge up the middle. That’s not a huge sample size, but tracking data on when the end run works and when it doesn’t will help address the question of whether or not it’s too good. As always, if you get the chance to play some games let me know how they go.

The Case Study: Material Science

One benefit of playtesting a game is that it points out issues, not just with the design, but also with the implementation–how the game physically works on a table. I’ve run into one such problem with Over the Next Dune, and I’d love to hear other people’s thoughts on how to solve it.

As a print-and-play game, all the pieces of OtND are simply paper. That’s great for ease of construction, but during play such a light material can be difficult to work with. In particular, it’s easy for terrain to get knocked out of position. The searchers have a similar issue, especially when they overlap. Getting everything back in its appointed place is inconvenient at best.

I’d like to make future prototypes out of something with a little more weight. However, more weight generally means more thickness–and if the terrain pieces are thick they can’t overlap and still lay flat on the board. Allowing the terrain to overlap adds some interesting possibilities to the game, so I’m not willing to opt for solutions that prevent it.

I’ve considered breaking the terrain into individual squares, and then using each terrain piece as a template. Under that system, one would put the terrain piece over the appropriate spot on the map, and then drop squares anywhere that doesn’t already have one. That could be interesting (and might allow for elevation . . . ), but it’s fiddly.

Much to my dismay, I’m a bit stumped on this. Are the individual squares actually perfectly fine, and I’m just being unreasonable? Is there a material I could use for the terrain that I haven’t thought of? Let me know what you think.

The Case Study: Testing, Testing . . . .

Not a great deal to report other than that I’ve played a few games with the new rules (3-person rescues, setting up all of the players in the center) and they’ve been working well. Three-person rescues are doing a good job of making splitting up a choice with consequences. Central setup has made shifting to one side or the other trickier. There was a Keystone Kops repeated-rescues situation in one of the games, so even with the new rules those are still possible.

I’m planning on sending some prototypes to testers in the near-term future, so I want to make sure the recent changes get a thorough going-over. As always, if you have feedback please send it along!

The Case Study: Limiting the “End Run”

While figuring out how to distribute player tokens in a small group, it became clear that the potential for the player tokens on the far left and right to run up the sides of the board was a problem. Although that strategy doesn’t work every time–in some games terrain or the searchers make progress along the sides of the board very slow–it frequently does, and it’s rarely very interesting. Having a boring-but-powerful strategy is terrible; it incentivizes players to play boring games, which can’t be right! Before going any further, this has to be worked out.

Facts: Since the beginning of Over the Next Dune, there has been a “flank” or “end run” strategy wherein the players on the far left and/or far right move to the sides of the board and then travel straight up. This strategy can be very powerful, for several reasons:

1. Searchers are often in the middle of the board, and spend relatively little time at the far sides. As a result, a player on the edge of the board is less likely to be caught.
2. The searchers spending little time at the edges also means that players there can push steadily toward their goal rather than spending time maneuvering.
3. Terrain is usually minimal on the edges, allowing rapid progress.
4. Searchers approach the far sides of the board from only a few angles, so players on the edges are not often surprised by a searcher catching them unexpectedly.

In addition to being powerful, the end run strategy is easy to implement. The players on the far left and right need simply move a few squares, and they fall into it almost by default.

Issue: How can Over the Next Dune be changed so that the end run contributes to, rather than detracting from, the fun of the game?

Rules:

1. Decisions players make should be interesting throughout the game.
2. Players must work together.
3. The game must admit of multiple solutions.

Thinking it through: The existence of the end run isn’t necessarily a terrible thing for OtND. Completely eliminating it would reduce the number of possible solutions. Furthermore, it can be fun for players to spot an approach and try to pursue it–making a plan and carrying it out can create a series of interesting decisions.

Right now, however, the end run is reducing options and interest. Its advantages mean that players on the ends tend to want to do it every time, and its simplicity means that it stops being interesting in a hurry. It needs to be toned done, either by making it weaker or by making it less available.

One obvious solution would be to plant some permanent terrain along the edges of the board. That would make it harder to progress up the board quickly, and encourage players to move toward the middle. However, that ends up reducing the options for the leftmost and rightmost player. Instead of having two options (move to the middle or end run), one of which is sometimes boring, they end up only really having one–to go to the middle. The end run becomes an ineffective choice; they could try it, but it’s probably just wrong.

It might be possible to have just enough terrain to discourage the end run while still making it a viable choice, but if it’s possible the balance would be exceptionally difficult to get right. I’d rather not go down that road.

Another possibility is to make it more likely that players will get caught going up the edges. However, as they are currently set up the searchers aren’t accomplishing that, and I don’t want to mess with their initial layout or movement. A lot of rules have been changing recently, and fiddling with the searchers just seems like putting too many variables into the experiment. There could just be a hard and fast rule on the subject–“if you move along the edge for three consecutive turns you get caught”–but that has both implementation problems (does a new searcher appear on the map?) and, well, it’s just an ugly bodge. Overall, the searchers don’t look like a productive place to look for a solution any more than terrain was.

There is, however, another lever to pull: the initial setup of the player tokens. Starting the player tokens more toward the center of the board adds challenge to the end run. It’s still good if players can make it work, but they will have to make a decision (!) to try it and work together (!) to shift safely to one side of the board.

In addition, grouping the players in the center of the board could reinforce that the players are a team. Although the rules say that OtND is a cooperative game, seeing the board with everyone spaced apart might give a different impression. Pushing the players together at the very beginning reinforces the message that acting in concert is part of the goal and part of the fun.

So far this solution seems like it deals with the problem and has a lot of upsides. However, there’s no way to be sure until it’s put to the test. Below is a new print-and-play file, with a map that has the players’ starting locations grouped in the middle. (The rest of the file is unchanged, so there’s no need to print out the rest if you already have.) If you get a chance, try the game with these new starting positions and let me know what you think.

Over the Next Dune – Print and Play – 3-21-14

P.S. I did fiddle terrain piece 4 a bit, but I wasn’t able to find a satisfactory way to make it easier to cut out while still being useful in play. I’m sorry!

P.P.S. This doesn’t change the arrangement of the players when playing with two to four–it only changes the coordinates where those players find their tokens.

The Case Study: Playing with 2-4

Armed with a rule from The Succession Wars, I feel we’re ready to figure out what happens when you want to play Over the Next Dune with a group of two to four. It’s a tricky problem that this past weekend really threw into sharp relief. A few people had schedule changes that left fewer players for testing than expected–and not having confronted this issue, I wasn’t ready with a plan. Clearly, it’s time to work this through.

Facts: OtND has five player tokens. When playing solo, a single player controls all of them. When playing with five players, each player controls one. Currently, the rules do not address what happens when playing with two to four players.

As an additional wrinkle, previous testing has demonstrated that not all of the player tokens have the same options. The tokens on the far left and right can move up the sides of the board; in some configurations of terrain and searchers this is very safe (and, therefore, not especially interesting). By contrast, the token in the center is generally in the thick of things from the beginning.

Issue: How should player tokens be allocated among the players when there are two to four players? (Technically, this is really three issues: what to do when there are two players, when there are three players, and when there are four players. Since much of the reasoning will apply to all three, it seems OK to collapse them into one for most of the discussion.)

Rules:

1. Decisions players make should be interesting throughout the game.
2. When one player needs to “sub in” for others, controlling multiple players’ pieces, the rules must ensure that the player cannot combine those pieces to become more powerful than the other players in the game.

Thinking it through: Rule (1) means that a single player should never control only the pieces on the far left and the far right. Although it’s unlikely that both sides of the board will be safe, it could happen that way. If it does, a player in charge of the two far pieces is liable to end up playing a boring game as both pieces just rush straight up the flanks. It’s OK if a player has one token making the safe “end run” and another token that’s doing interesting stuff, but if both are safe the player is apt to be bored.

Rule (2) means that a single player should not control tokens that start out next to each other. Such tokens are likely to be in a position to help each other out; as a result, that player will be able to keep his or her tokens safe without having to confront the rules limiting communication with other players. Giving one player total coordination to make saves, while everyone else has to struggle to work together, is clearly giving that one player a great deal of extra power..

With three players, the rules allow this configuration:

(player 1) (player 2) (player 3) (player 1) (player 2)

In other words, player 1 controls the tokens starting at (1,2) and (1,14); player 2 controls the tokens starting at (1,6) and (1,18); and player 3 controls the token starting in the center. This arrangement gives each player a relatively central token and avoids anyone controlling tokens that are next to each other. Player 3 only gets one token, but it’s the one in the center which presumably has the most options. He or she will get to make fewer decisions, but they’ll be interesting ones.

For four players we have to substitute player 4 in for one of player 1’s or player 2’s tokens. This is a little awkward, because we either have to give player 4 one of the end tokens (which will sometimes mean an easy game) or leave player 1 or 2 with only an end token (saddling them with the same problem).

Lacking a really satisfactory option here, I’m going to assign player 4 to the far right token at (1,18). It’s slightly closer to the center than the far left token, and in theory should therefore have a slightly greater chance of getting into the mix early. The arrangement is:

(player 1) (player 2) (player 3) (player 1) (player 4)

When there are two players I think this arrangement works:

(player 1) (player 2) (player 1) (player 2) (player 1)

Player 1 has the two tokens that might have unexciting games, but also has the center token that has a lot to do. Neither player has adjacent pieces, so even though there are few players they will still have to coordinate.

That solves the problem we started out with: there are now rules for playing with two to four players. However, working this through has emphasized a new issue. The far left and right tokens aren’t working as well as they need to. It’s awful that there are tokens whose positioning can lead to uninteresting games for their players. I’m going to put some thought into what should be done about them, and we’ll discuss it more next time.