careful interpretation, as there are many biases that unwittingly impact player’s stated opinions.
CHOOSING THE RIGHT METHOD Research methods fall into four broad categories: Observation Watching players play provides an unfiltered view of real behaviour. These can be conducted live (in person or remotely) or through watching recordings of previous sessions or streams. • Strength: Reveals authentic player behaviour in a realistic context.
• Weakness: It takes time to watch people play. Observation based playtests are best for research objectives that are focused on understanding behaviour.
Interviews Directly asking players about their experiences helps uncover motivations, misconceptions and thought processes. These are best conducted live during or immediately after playing a game to minimise bias. • Strength: Provides detailed insights into player perspectives and explains the behaviour you’ve seen.
• Weakness: Players may misremember or misrepresent their behaviour, and so validation from other data sources helps get to the ‘truth’.
Interview based playtests are best for research objectives that are focused on understanding opinions.
Analytics Use telemetry to automatically track player actions to measure trends at scale. • Strength: Provides large-scale quantitative data, representing your real player base.
• Weakness: Lacks context for explaining why behaviours occur.
Analytic based playtests are best for research objectives that are focused on measuring behaviour.
Surveys Ask qualitative and rating questions to collect player opinions on experience, difficulty, pacing, and more. When we’re worried about leaks, this might require running multi-seat playtests to get this data in a controlled environment.
• Strength: Useful for benchmarking across different levels or games.
• Weakness: Often lacks enough depth to explain player behaviour, making it difficult to identify what action to take to address problems.
Survey based playtests are best for research objectives that are focused on measuring opinions.
COMBINE METHODS FOR STRONGER CONCLUSIONS You’ll notice that your current research objectives probably don’t all map to the same method. Combining methods can be a powerful way to get a complete picture of the player experience, and give your team the clarity they need to fix the issues identified. No single method provides a complete picture, and each method has weaknesses. Combining them helps overcome these issues - some of the most common methods I use myself include: • Live Observation + Survey + Interview: Watch people play through a section of the game, collect ratings via a survey, and finish with an interview to understand the behaviour and ratings we saw. This is popular when optimising the FTUE and tutorials for games.
• Analytics + Observation: Identify problem areas with analytics and watch recordings of people playing through that section to explain the issues. This is popular in post-launch games for optimising retention.
PRAGMATIC PLAYTESTING We’re always operating under a lot of time and budget pressure, so sometimes the most appropriate method might not be possible. Understanding the characteristics of each method (as described above) can help make an informed decision about trade-offs and de-risk, Personally, I find teams get the most benefit starting
with watching someone play and occasionally asking questions. This method provides rich behavioural data and allows you to ask probing questions to understand the behaviour that is occurring, making it one of the most reliable ways to gather actionable insights.
Steve Bromley runs playtests and helps teams learn how to integrate user research into their development process. For a more detailed guide on playtesting methods, check out the free book Playtest Plus on
gamesuserresearch.com
April/May 2025 MCV/DEVELOP | 25
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46 |
Page 47 |
Page 48 |
Page 49 |
Page 50 |
Page 51 |
Page 52 |
Page 53 |
Page 54
