Developer for Nuclear Dawn mod (May 2005-Dec 2008) :: QA Tester for SEGA Europe (Dec 2007-Aug 2009) :: Production Tester for Splash Damage (Aug 2009-)

Report article Mod Tester's Handbook

A reference guide for testing mods. Includes info on how to find bugs, how to narrow down the steps to reproduce and how to write good bug reports. Also includes a 'game development terminology' reference section.

Posted by Crispy on Jul 25th, 2008 digg this super bookmark Page 8 of 11    
Intermediate QA/Testing.


Writing a Bug Report: Communication

Exhaustive, clear, concise: a bug report is all these things

The most important quality to look for in a tester is an ability to communicate important information clearly to your developers in a professional manner. If they are really good at finding bugs but can't explain the bugs clearly to the devs they will end up wasting a lot of time. If they come across as very negative with their feedback they will create disruption within the team. There are a few things to keep in mind when writing a bug report: you need to be precise, you need to keep it relevant and you need to stay neutral.

Precision
Accuracy is key here. Don't make any claims in a bug description unless you are 100% sure about them, the last thing you want to do is give information that could be misleading. If you have a gut feeling about what is causing the issue, use phrases like 'it seems to be', 'this may be' or 'perhaps this could be'. Don't ever make assertions if you aren't completely certain, it only wastes time. If talking about reproduction rates, give actual numbers instead of percentages; 3 out of 15 is a much more illuminating figure than 1 out of 5, even though both could be described as a 20% reproduction rate. Precision also extends to terminlogy: when describing bugs and how to reproduce them, you should be using the exact names the designers have given to characters, weapon, moves and locations. If an area of the game is introduced as the 'Vineyard' in the game, do not refer to it as a 'Farm' or 'Field' or fellow team members will be less likely to find your bug in a database.

Relevance
Don't include any extraneous information. Your job as a tester isn't over until you have narrowed down your steps to creating a 100% reproduction rate with the minimum amount of steps possible. Everything you include in your bug report has to be relevant. On the other hand, don't make any assumptions about what is and what isn't relevant if you are unsure. If you have found that a graphical issue that can be reproduced in both resolutions of 1024*720 and 800*600, make sure this information is in the bug report (don't ever forget to mention any conditions under which the bug cannot be replicated, this information can hold important clues). If in doubt, include extra information as a note in the bug.

Neutrality
Stay neutral. Give as objective an account of the issue as possible. Humour or sarcasm can be misinterpreted to create ill feeling and negative feedback is an unhelpful approach. If you feel you need to give a personal opinion try to distance yourself from your statement to make it sound more humble and less self-righteous. Phrases that show consideration for the end-user instead of over-emphasising personal preferences will be much more readily received by the developers than personal attacks or overbearing judgements. E.g. "It is this tester's opinion that the control scheme is unintuitive and may cause the end-user considerable frustration" sounds better than "The controls are fubared".

Comments
AgeNt_
AgeNt_ Jul 25 2008, 1:59pm says:

very useful for beginning moders! gj

+1 vote     reply to comment
Sigma
Sigma Jul 25 2008, 4:09pm replied:

and for experienced ones to :)
required read for every modder :)

+1 vote     reply to comment
OkeiDo
OkeiDo Jul 25 2008, 7:21pm says:

Interesting read, I myself have tested a few mods now and still am so I'm quite familiar with some parts in this tutorial. However, it felt like some parts were mostly for single player games/mods but it's good that you managed to cover all parts. Good job and keep your tutorials comming! I will definitely show this to my fellow testers.

+2 votes     reply to comment
Crispy
Crispy Feb 16 2009, 3:23am replied:

Hi! I know it's some time since you made this comment, but I've updated the Bug Hunter section with some ideas for multiplayer testing anyway.

+1 vote     reply to comment
Shiggy
Shiggy Jul 26 2008, 1:57am says:

This is a very good read. I have been testing games for a while, and this clearly explains all of the characteristics of a bug that a game tester must explain to the developer. This is a must read for any developer/game tester, since it shows how much organization can help you in the bug-fixing process.

+1 vote     reply to comment
Gibberstein
Gibberstein Jul 26 2008, 10:02am says: Online

I thought page three was a bit too deep and overcomplicated for most mod teams - it doesn't need to be so formal in a small team. Might be valid for one of the few really large mod teams though.

The rest of it is spot on though - if all bug reports were done to this standard I'd be a very happy developer :)

+2 votes     reply to comment
Crispy
Crispy Jul 27 2008, 10:11am replied:

Funnily enough this piece started off as part of a bigger tutorial, which I then decided to split into one for QA testers and one for QA Leads. When I wrote the introduction to what will now be the management tutorial, I put in a warning that small mod teams don't need to exhaust their efforts on fancy bugtracking software if they're only going to have 500 bugs in their database. There's no point writing up playthrough testplans if all you're changing is some weapon models and variables.

The same is true of testing to a certain degree. As far as a tester is concerned, the main points to focus on whether you're a big or small mod fall into two areas:
1) Bughunt actively, don't wait for bugs to come to you
2) Communicate information with the whole team in mind, whether it's a bug report or a fix.

What parts from this tutorial you choose to apply to your own testing is down to the scale of the project. Ultimately the QA Lead should be setting the benchmarks for your mod's bugging practise, which is what the next tutorial will look at.

+1 vote     reply to comment
JohnBart
JohnBart Jul 26 2008, 3:47pm says:

Thank you for making this tutorial Crispy. :-)
I have already sent a link for our beta testers, as there is a LOT to learn from this article.

Personally, being the beta team leader of our mod, I can't wait to read your upcoming QA management tutorial. ;-)

+1 vote     reply to comment
Mr_Cyberpunk
Mr_Cyberpunk Jul 27 2008, 6:15am says:

OH MY GOD this is so awesome.

Looking forward to your QA Lead guide

Good work Crispy.

+1 vote     reply to comment
Crispy
Crispy Jul 27 2008, 10:17am says:

Thanks for the kind words.

If you have any D class bugs of your own, I am taking all feedback on board to work on improving the tutorial and making it more relevant to more teams. Based on the comments so far, I'll be looking at adding in more stuff relevant to multiplayer testing and also trying to make it clearer what is applicable to smaller mods.

+1 vote     reply to comment
Forceflow
Forceflow Aug 1 2008, 8:53am says:

Very interesting read, Crisp.

+1 vote     reply to comment
Post a Comment

Only registered members can share their thoughts. So come on! Join the community today (totally free) and do things you never thought possible.

Level
Avatar
Avatar
Offline Since
Nov 14, 2009
Country
United Kingdom United Kingdom
Gender
Male
Age
25
Member Watch
Track this member
Tutorial
Browse
Tutorials
Report Abuse
Report article
Bookmark
Digg Super bookmark