In this blog post, I'll explain the review system I wish to use for my reviews on indiedb/moddb/Desura, both for future reference for myself as well as for anyone who's interested.
First of all, I don't want to just be handing out 10/10s or 0/10s, and here's why: From what I've seen, most scores in here are 10s, and the community is giving high scores very easily. While it's nice to cheer the developers by giving perfect scores, it's not very helpful in the long run. The worth of a 10/10 diminishes over time, if most reviews give that score, and an 8/10 is even considered bad in some cases. It's of course difficult to give a proper score, since a plain number doesn't say much about its actual meaning. Should a 5/10 be considered border-fail? Or is it an average?
Scoring system
To solve this problem, I'll write down a description for each score, so that every number has a meaning. IGN has a nice score description, so I think I'll just use that. Here it is:
10 - Masterpiece
9 - Amazing
8 - Great
7 - Good
6 - Okay
5 - Mediocre
4 - Bad
3 - Awful
2 - Painful
1 - Unbearable
0 - Disaster
As I understand this scale, you can't have masterpieces every day. I consider a true masterpiece something that truly stands out from the competition in every way and is either flawless, or has so little and negligible flaws that its content more than makes up for it. This is a game that people will be talking about for years to come, and will stand as a milestone for every game coming out after it.
"Masterpiece" frequency: 1-2 per year, maybe less.
An amazing game is also standing out of its competition, and should surely be a candidate for Indie of the Year. Amazing games do almost everything right and are the reasons I play games.
"Amazing" frequency: 5-10 per year.
Great games are thoroughly enjoyable, and are worth to reach a wide public. Many gamers are listing these games often as one of their favorite games at the time, and any studio developing a great game should be proud of it.
"Great" frequency: 2-3 per month.
Awful, painful and unbearable games should've never been released. They were either rushed or the concept was plain wrong to begin with. They're games that players have to force themselves to play and are another reason for Roger Ebert to not consider games an art form. They thankfully don't appear more than a few times per year, and when they do, they are quickly forgotten.
Disaster games are, put simply, a shame for the developer and the gaming community as a whole. The developer should take a break from making games for a long, long time, and give customers their money back. Disaster games are thankfully rare, but there's at least one every year.
Review
Having dealt with the scoring system, here are some thought about the actual review.
A plain score is meaningless for the developer. It's like having a blindfolded man and saying "hot" or "cold" instead of taking the blindfold off of him and telling him where to look at. Reviews should never serve as a means to let off steam, but should always be constructive criticism. The game's strengths should be praised, and its weaknesses should be addressed with the severity each of them has on the final game. If possible, suggestions should be made to overcome the game's weaknesses. That's another reason why empty, perfect "reviews" are not actually helping the developer.
The personal opinion of the reviewer can never be hidden completely, and shouldn't be; but the reviewer should be open-minded enough to realize that there are people with many different tastes. It's actually better to say "I didn't like this or that", than bashing it in general. Some things are thankfully objective, like bugs or the FPS count, but aspects like the art style or the game setting are up to every individual gamer and should be handled accordingly.
Lastly, the game version should always be included. This tells people what state of the game you're referring to. It's unfair to treat a finished game the same as its alpha version. There should also be mentioned what parts have been played (campaign, multiplayer), and if the game was the full game or a demo.
What do you think? How do you review your games? Do you have any suggestions for me? Feel free to comment below.
