Chris and Dave here. This week we start our first internal playtesting for Severed! A DrinkBox mantra is to playtest early and often. Basically this means harassing family, friends, and amazing volunteers to play our game.
The average playtest is streamed to the entire studio so many members of the team will be able to watch in real time. Despite this, we feel very strongly about the benefit of logging playtest data – this is information recorded to the Vita in the form of a text file while somebody plays Severed. Log data is really good at giving the macro view of a playtest session, which can help us as a studio tune the game for maximum ultra uber fun.
So what does our log data look like? Initially, tens of thousands of lines of this – taken from a recent playtest by one of our artists:
Raw playtest data
But after we import this into our analysis tool, it runs various statistics on the data and spits out something much more readable to give a high-level view of the play session:
The ‘level summary’ section of our processed logs
We have a target in mind for time required to complete each level in Severed to ensure people get a good value for their purchase. Immediately we see that this level took around 55 minutes to complete, which is actually on target. That’s great.
As I look closer at the data though, I see the majority of monster parts collected was Coral and Tentacles. Coral is an early level item which is appropriate for the Father level, but the Tentacles is not. Tentacles are used for later game upgrades, and therefore should be collected later in the game. This becomes a task for a Designer to investigate and balance out appropriately.
Further down the log data we look at player deaths – perhaps the most important piece of information. Recording deaths allows us to eliminate difficulty spikes, a great bane for many developers. You can see that node (B1, 12, 1) has 12 deaths, which is very high compared to the next highest at node (B1, 9, 1) with 3 deaths. For Severed, the disparity between these two difficulties is too spikey for our liking and we actively review the fight that takes place in (B1, 12, 1) to determine why our player died so often.
Node (B1, 12, 1)
Even if we’re seeing a lot of deaths here, the solution isn’t necessarily to just nerf this particular fight. Also in our handy-dandy log analysis is a summary of how each individual battle went, to give us more detail to diagnose the issue…
On this attempt, despite parrying a lot, using a spell and holding out for two minutes, he still died, which is quite painful.
And we also produce a general breakdown for each enemy, which collects data across all of the battles in which that enemy appears and helps us try to guess how often they’re an issue:
We have to get those Bad Barrys down.
We might notice by looking through here that Strongarm is one of the top enemies killing the player, and that it does higher damage on average than most; a possible red flag that it’s the culprit, at least in this battle’s particular mix of enemies.
We’re still developing these tools, and trying to get an accurate picture of what’s really going on through abstracted stats will always be tricky, but by coming at it from these diverse angles – plus reviewing the recorded video and, of course, talking to our intrepid playtester – we end up with a powerful set of tools for analyzing what’s working and what isn’t in Severed.