shmups.system11.org

Shmups Forum
 
* FAQ    * Search
 * Register  * Login 
It is currently Tue Dec 18, 2018 4:45 pm View unanswered posts
View active topics



Post new topic Reply to topic  [ 4 posts ] 
Author Message
 Post subject: Good feedback questions for level testers?
PostPosted: Tue Sep 11, 2018 9:13 am 



Joined: 15 Apr 2006
Posts: 298
I'm thinking I should start having my levels playtested. I would post test builds in trusted forums and anyone could test at their leisure and fill out a feedback form.

Questions so far:
  • # of times level played vs # of times level cleared
  • Prior shmup experience level
  • I stopped playing because...

    Agree or disagree:
  • I could clearly make out enemies and their attacks, including those attacking from the flank or the rear, and some ways to avoid them
  • Enemy attack intensity was appropriate for this point in the campaign (_th level out of a planned _)
  • I had the right amount of resources (options, bombs, etc) to fight the enemy, without starting on a previous level

Any other good questions? Any of these not so good? How many questions is too many?

Preview:
https://goo.gl/forms/x9fSLB0BQwgqTACh1


Top
 Offline Profile  
 
 Post subject: Re: Good feedback questions for level testers?
PostPosted: Fri Sep 14, 2018 5:05 am 



Joined: 16 Jul 2018
Posts: 12
Just wanted to make a couple suggestions based on my own experience doing this sort of thing.

1) The question 'Prior shmup experience level' isn't what I would consider well formed. It's too open ended, vague, and somewhat meaningless. Something more like 'How often do you play games like this one?' along with provided cues like 'All the time', 'Occasionally', 'Almost Never' would be easier for people to answer and for you to comprehend the result.

2) I certainly understand your desire to get a lot of questions answered but avoid the temptation to throw the whole list at every player that comes along! There are a couple of reasons for this. a) Nobody actually wants to answer your questions. Like, ever. They might be keen to give opinions and make comments but filling out what feels like a test just isn't fun. b) You want to have focus. Pick one aspect that you are specifically curious about and then cull your list of questions down to the most important ones related to it. For example: if you want to know how players feel about item drops, ask them things like, 'Do weapons drop often enough?', 'Should they drop more often?', 'Are they interesting?', 'Which was your favorite?', 'Least favorite?' You don't want to overload their thoughts with a bunch of seemingly unrelated topics and if you keep it focused you'll likely find they are more tuned in to that after thinking about it just a little and are now ready to give more detailed information in that space you've inevitably left as 'Additional comments and thoughts are welcome here!'

3) You probably already know this but if a picture is worth a thousand words, then a video stream is worth billions. If you can convince someone play the game while you silently observe you'll get so so much more information than any questionnaire could provide. If you can record the play sessions that will help a lot too because you'll likely miss many details the first time, plus being able to re-watch certain clips will allow you to break down user actions and deduce what kinds of things you can/should/need to change.

Best of luck!


Top
 Offline Profile  
 
 Post subject: Re: Good feedback questions for level testers?
PostPosted: Wed Sep 19, 2018 9:58 am 



Joined: 19 Jan 2009
Posts: 645
Location: stuck at the continue prompt
Sluggy wrote:
3) You probably already know this but if a picture is worth a thousand words, then a video stream is worth billions. If you can convince someone play the game while you silently observe you'll get so so much more information than any questionnaire could provide. If you can record the play sessions that will help a lot too because you'll likely miss many details the first time, plus being able to re-watch certain clips will allow you to break down user actions and deduce what kinds of things you can/should/need to change.

A good idea, but with some technical effort you can do even better: in a alpha or beta version you can automatically record input replays of every single game and have them sent to you, instead of recording a few huge video files, receiving them only if everything goes well, and potentially wasting time watching them.
Using replays, more information than what can be seen in a video is available, and you should be able to automate some important data analysis.
Simulating game states throughout a game allows answering statistical questions like
  • Do players have the expected stock of powerups, bombs etc. when they reach some milestones, like bosses? If they have less, were the missing items lost after dying, expended early or not picked up? If they have more, where is the exploit?
  • Where in the level do players get killed most often? By what possibly unfair enemy attacks?
  • Are the players attacking enemies with the "right" weapons?
  • Which paths, in case the next level can be selected by the player like in Gradius or Deathsmiles, are more popular? Which ones are associated with higher or lower scores or with significant events (e.g. running out of bombs)?
This way you could watch only a few playthroughs in full, for inspiration and clarification, rather than to collect raw data. You know what replays contain interesting occurrence, and when.


Top
 Offline Profile  
 
 Post subject: Re: Good feedback questions for level testers?
PostPosted: Fri Sep 21, 2018 10:27 pm 



Joined: 16 Jul 2018
Posts: 12
Ixmucane2, definitely a good idea! Though I'd argue it's not necessarily better, just a different toolset. To paraphrase the developer of Anti-Chamber, 'Analytics doesn't measure player frustration.' People's actual emotional reactions to your game, good or bad, can't be replaced by statistics.


Top
 Offline Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 4 posts ] 

All times are UTC


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
Space Pilot 3K template by Jakob Persson
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group