Mod Tester's Handbook tutorial - Crispy

Bug detective

What to do if you find a bug

Communicate:
As a tester, the first thing you need to do when you find a bug is to tell everyone else about it to create a general sense of awareness. This will mean other testers are more focused so if they do find the same issue, they will be able to provide more information on how to attempt to reproduce it. Reproduction is the second course of action. Try to recreate it by doing exactly the same thing you did before. If this still produces the same effect, you need to try to narrow down your steps by changing one condition but keeping the rest the same.

Information gather:
The best first step is to try to note down as much information about the incident(s) as possible. When you write a bug you will need to include most of this information in any case, what you need to do next is eliminate any information that isn't specific to the bug. Remember that hardware and software settings are also 'conditions' for the bug. It could be that the game is asking your graphics card to do something it can't, it could be that you have on-screen text turned off and the game is trying to give you a tutorial message that conflicts with this option setting.

Eliminate:
The greatest detective of all time once said: "If you can eliminate all the other choices, the remaining choice, no matter how improbable, is the answer." Once I spent 4 hours trying to figure out what was producing a 100% reproducible crash in a game. When I tried it with another game disc it still occurred, but when I tried it on a different machine it didn't occur. The only conclusion left was that the hardware (the console) was unreliable. Usually we go into testing assuming that the tools we use are there to help us find problems, but as a tester you have to consider that sometimes they can be part of the problem themselves. Ever since that experience I always make sure that I check in the early stages that hardware isn't the cause of the issue, I'd advise you to do the same, especially when talking about PCs that have many different pieces of hardware to keep track of.

Eliminate conditions beginning with the most general. One of the first things you should try if you can do so easily is to try to reproduce using the same conditions on a different machine. Other general conditions could be to see if the same bug can be reproduced on a different level, with a different weapon, with a different class, against a different enemy. If it's a graphics issue see if it occurs on a different resolution. Try to ascertain how exact the steps need to be for successful reproduction.

Reproduce:
Reproducing is really part of the elimination process, but even when you think you have the answer you really need to remember to try to get another tester to reproduce the problem using what you have written as a guide. Sometimes you might have missed an important bit of information that will be uncovered when the other tester fails to trigger the bug using your steps to reproduce. Sometimes this missing information is the key to what is causing the problem in the first place.