Measuring Success in Game Development

By Katy Farmer / Product, Use Cases, Developer
Jul 31, 2018

I’m still on my personal mission to explore and understand the value of metrics as a fundamental part of tech. We’ve entered an age of instrumentation, and I feel like I’ve been left behind. I’ve seen metrics used to monitor complex systems, analyze behavior (including that of software, hardware and humans) and track the changes in machine learning algorithms. Sensor data can tell me when there’s too much carbon monoxide in the air, or if my online store crashes on Black Friday. Metrics can help keep factory workers safe and track my grocery delivery. Those are all pretty dang impressive, and it’s easy enough to see the value when other people explain their metrics to me.

I decided that the easiest way for me to explore the value of metrics in my own work was to instrument an application that had value to me. Enter “Paper Pirates”: a browser-based, Node.js game I built that has no purpose but to be fun (find the repo here). There is no winning “Paper Pirates”—the only winning move is not to play.

I chose a web game for a few reasons: 1) I love games. They’re so much better than reality. 2) Game development is really hard. Even in a simple game like this, there are a lot of opportunities for glitches and unexplained behavior.

Games are a goldmine of metrics, and there are plenty of stats I can gather to make sure my app is running as expected. This is the heart of DevOps, but I want to explore a slightly different angle. In this article, I’m going to focus on the ways that metrics helped me tune the game performance and improve gameplay. Let’s go!

Gathering metrics from “Paper Pirates” is easy; I used the InfluxDB Node.js client. Essentially, I listen for certain events, and when they occur, I send a value of 1 to InfluxDB. In this case, I don’t need a specific value—I just need to know that the event occurred and when.

All of the interactions with InfluxDB live in index.js. Looking at the code, you’ll see I’m gathering metrics to help build leaderboards, measure missile accuracy and overall player details. For now, we’re going to focus on this block:

socket.on("enemyfire", results => {
    client.writeMeasurement("events", [
      {
        tags: { sessionID, gameID, enemyID: results.enemyID, event: "enemyMissile" },
        fields: {
          enemyFires: 1,
        }
      }
    ])
  })
});

One of the most important parts of a game is player interaction with the world, which is, quite honestly, one of the last things that occurred to me—until I played the game the first time. This is how the enemy ships fire:

export const enemiesFire = () => (dispatch, getState) => {
  const { enemies } = getState();

  _.each(enemies, enemy => {
    const roll = _.random(4000);

    if (roll <= 10) {
      Client.emit("enemyfire", { enemyID: enemy.id })
      dispatch({ type: "ENEMYFIRE", payload: enemy });
    }
  });
};

There’s a lot going on here. Essentially, we’re dealing with the probability that an enemy will fire. A random number is chosen between 0 and 4000, and only when it is less than 10 will an enemy fire a missile. When I wrote this the first time, I had no idea what numbers would work, so I guessed. We refer to the results of this first experiment as The Void.

<figcaption> The place where you’re alone with your mistakes</figcaption>

The enemies were firing so often that just generating the missiles caused everything else on the canvas to lag. Then the player died immediately as a boatload of missiles slammed into them.

This led me to change the number in the rand function a few times until I found something that didn’t kill the browser. However, one of the toughest parts of game development is figuring out what feels right. Things like how responsive the controls are, how quickly the sprites react or even how precise the collision detection is all add up to a player enjoying (or not) the experience. So finding the right rate of enemy fire was important to me, even if it was hard.

Enter metrics! Revisiting the first code snippet, I marked each enemy missile as an event that increments a count being sent to my instance of InfluxDB. This allows me to track meaningful aggregates correlated with the random numbers I’ve selected in my code.

In this way, metrics drove the improvement of gameplay in “Paper Pirates” and made me a more informed developer. The first time I wrote the random firing, it looked like this:

_.each(enemies, enemy => {
    const roll = _.random(1000);

    if (roll <= 10) {
      Client.emit("enemyfire", { enemyID: enemy.id })
      dispatch({ type: "ENEMYFIRE", payload: enemy });
    }

Tracking the events, I determined that the reason it destroyed the entire game was that it was firing about 3,500 times in 10 seconds. “Paper Pirates” just couldn’t handle that kind of load. As I played the game and tracked the fires, I determined that about one enemy fire per second felt best to me and made it possible for the player to survive for longer than a minute.

<figcaption> The finest vessel on the sea</figcaption>

Another reason I needed aggregates is that there are multiple enemies on screen. The game is currently limited to four enemies at a time, but they all fire at slightly different rates, which, in the simplest terms, means I have no idea what’s going to happen. Gathering the number of enemy fire events eases my personal mental burden of trying to figure it out. It also allows me to perform all kinds of aggregates on it, although I’m mainly concerned with the average.

Understanding the value of metrics isn’t going to be the same for everyone. Yes, I want to know that “Paper Pirates” is up and running and metrics and events can help with that. But I also want to know that when it is up and running, it’s also the best it can be.

Being able to fine-tune gameplay is one part of creating a game that people actually want to play, and measuring how that gameplay changes over time as my code changes is a way that I can become a better game developer.