Desura reviews: It's a 10 - or is it really? Thread

I think that the review feature on Desura is a wonderful way to get your review seen by both the developer and people who might be interested in buying the game. But I feel that the system is misused by unrealistic scores or outright lying.

A lot of games - a LOT of games - are getting many 10-score reviews. I wont argue if any particular game is "the best game ever" but the amount of games that are actually scoring 10, the highest score possible, is quite staggering. In my book, 10 is reserved for games that are perfect, or near perfect. Minor bugs aside, these games are the dope. I can count on one hand the amount of games I have played that I would give a solid 10.

I know that different people like different kind of games, and different games should be evaluated independantly - you wouldn't penalize Portal 2 because the graphics are worse than Bioshock, because Portals graphics fit the game and does what it needs. You don't deduct points from To The Moon because it doesn't have turn based combat.

If I look at the latest reviews on Desura (https://www.indiedb.com/reviews), 61% of all reviews posted gives a 10. Wow, that is amazing - we must be sitting on some of the best games this year! Do you see what I mean?

But using unrealistic scores does nothing good for anyone. It might help motivate a developer to continue working on the game - or it might be a developer trying to pad his numbers. It might also be someone trying to get on the developers good side, or simply someone trying to bump their rank on Desura for a bit of an ego boost.

But if I buy a game, thinking its a solid 10, then I have some really big expectations to the game. If they are not met, then I will tell my friends that "The game was ranked really well, but it suuuucked". Suddenly, we are discrediting the value of rankings on Desura altogether. And now, it's not about getting an accurate representation from the masses of Desura, it's suddenly about getting as many 10's as possible. I've posted a few reviews on the site, and I have felt bad because my "8" pulls the game down - when in fact I feel it is an accurate representation of my experience with the game.

So please, consider how you rate when you rate. I am not saying that my way of reviewing games is in any way perfect, but I do feel that very few games deserve a 10. God, I wouldn't want to play the games you rate 1.

In my book, any game above 5 is a playable and enjoyable game. A game doesn't have to be 10 to be awesome.

It's an opinion, so be honest. People didn't argue when CoD got purposely slammed on Metacretic, or Gamestop.com, just because people weren't happy with something besides the game.

Personally, I don't like the 1-10 rating, it's to wide a range and people put the school game passing system to it (IE a 7 or 6 & under = fail). I like X-Play's system: 1-5. 5 = awesome, everybody will most likely like the game, 3 = it's decent but people not fans of the genera won't like it & 1 = it really is junk.

This kind of thing happens everywhere, not just here.

I know, but I figured we could talk a bit about this mentality. Currently, ratings aren't really worth much.

A 1-5 scale does make more sense, because you could attach general guidelines to the game.

Similar to a 1 - 5 rating system could be to incorporate a sort of star rating, that to me seems like quite a nice idea. As I would agree that 1 - 10 is just a far to bigger range.

That aside though I think people get into the simple mentality of "I liked it therefore it's a 10" and "this sucked it's a 1". I also think this is driven somewhat by the fact that usually when a game is incredibly good you almost want to support the game and tell people about it and in the opposite case give it a 1 if it was bad in order to vent your anger. So usually you just end up with the extreme voices rather than an equal number of all review types, at least In my opinion I've found I don't usually bother much with ratings and reviews for games I would describe as "meh". Obviously this could just be me but it does explain how you end up with these extreme 10s appearing on most games.

I think the only thing you can do to combat this is to provide a simpler rating system rather than offering people a buffet of numbers.

Edited by: KurlyWurly

There are never any real honest ratings on Desura or any of our sites. But when a game gets large enough the votes kinda sort themselves out. Reviews that people agree with are far more important to the system then those that just rate it whatever and are done with it.

Having said that it is either rathed really high or really low. There is never a happy middle.

Do you think we could change this mentality by going with a 5 point rating system instead, and showing people "What the rating means" when you hover over it?

Would like to see a complicated review system with simplistic options. Not the large-gap-score-system-that-is-skipped-anyway of 1-5 or 1-10.

A poll-based review system. Something like:
1. Did you fully enjoy gameplay? yes/no
2. Do you really like game graphics? yes/no
3. Did you fully enjoy written dialogues and voice acting? yes/no
4. Did you fully enjoy soundtrack? yes/no

... and so on. Without the "general", generalised ("did you enjoy the whole game?") option.

Fully and really are keywords here. And simplistic, quite "primitive" options - a simple yes/no, without a "full/half" score system.

Then the community % percent score of each game feature (gameplay, game balance, feature-richness, graphics, dialogues, soundtrack, sound effects etc) would be automatically calculated by the system. Yes for 1 point, no for 0. So completely binary.

I think what feillyne said is quite a good idea !
something like
graphics : rate 1 to 5
story : rate 1 to 5
sound effects and music : 1 to 5
gameplay : 1 - 5
etc : 1-5

and then take this and make it average

Guiboune wrote: I think what feillyne said is quite a good idea !
*Points based system*

Doesn't that beat the point of feillyne's system? Adding the 1 - 5 scores I mean. I actually like the idea of only having binary choices and I fully agree that a 1 - 10 range is too large.

I usually find it hard to pick the 'right' score, that's why I usually go for 8's and 9's for the games and mods I really enjoyed, regardless of the technical aspects (graphics, sound etc.). I don't play a lot of games/mods I did NOT like, but for those subjects it's even harder to pick a fair score. With a 1 - 5 range it would be a lot easier for me to rate games and the binary system sounds even better, but of course I will manage with the current system

The most important part of reviews is the review itself, not the score. I honestly don't even look at the ratings, since they are just numbers based on opinions.

Personally the idea of adding things like questions and specific ratings for different parts of the game just makes it more complex. Imagine trying to rate the graphics of Minecraft? Though you could argue this style is purposeful so it's okay and therefore someone may give it a max score for graphics which effectively brings us back to the original problem of people just rating 10s.

What we really need is a simple system based around 5 numbers or preferably as I would like a star based system that way people would be more likely to go for the lower numbers, as I said and Henley did people always go for the extremes so what's the point in having all of those extra numbers in the middle?

Metalspy wrote:

Guiboune wrote: I think what feillyne said is quite a good idea !
*Points based system*

Doesn't that beat the point of feillyne's system? Adding the 1 - 5 scores I mean. I actually like the idea of only having binary choices and I fully agree that a 1 - 10 range is too large.

Well it's just because a yes or no question is the extreme of the extremes. how does someone answer "kinda" to that ? choosing no would put it as if it sucked and choosing yes would put it perfect.
and how would a score be calculated on this ?
anyway, I do feel like 10 points systems is a bit too much

KurlyWurly wrote: Personally the idea of adding things like questions and specific ratings for different parts of the game just makes it more complex. Imagine trying to rate the graphics of Minecraft? Though you could argue this style is purposeful so it's okay and therefore someone may give it a max score for graphics which effectively brings us back to the original problem of people just rating 10s.

What we really need is a simple system based around 5 numbers or preferably as I would like a star based system that way people would be more likely to go for the lower numbers, as I said and Henley did people always go for the extremes so what's the point in having all of those extra numbers in the middle?

Reviews should be made in respect to the genre and the style. You wouldn't score an 8-bit game a 1 just because the graphics are made in 8-bit. Then you consider the graphics in relation to the style the developer is trying to convey.

Qieth wrote: Reviews should be made in respect to the genre and the style. You wouldn't score an 8-bit game a 1 just because the graphics are made in 8-bit. Then you consider the graphics in relation to the style the developer is trying to convey.

Because that would make them awfully more complicated to implement on the site and because people will start to ask for stuff like "is it bloody enough" when it's a hack and slash and rate their games on a 2^16 points system because it has that much weapon possibilities in game "in respect to the genre"

Everyones bringing great points to the table here, but I fear that how can one change the mindset of people giving it 10/10 because they liked it etc.
People have different views on "rating systems" mine which is similar to yours Qieth that I only have a few games I would consider 9/10 but for others anything below 7 is bad and soforward.

Maybe feillyne's idea can be looked into?

Qjeth wrote: Reviews should be made in respect to the genre and the style. You wouldn't score an 8-bit game a 1 just because the graphics are made in 8-bit. Then you consider the graphics in relation to the style the developer is trying to convey.

How will you know what the developer is trying to portray? Unless they tell you however that seems fairly unlikely which means that effectively everyone should give a game a high rating for it's graphics as it's impossible to differentiate intention from purpose.

A simple 1 - 5 system is all that's needed to decide whether overall (combining everything like graphics etc into one) it was a good game. P

Wouldnt just changing the rating system from 1/10 to 1/5 just make the same people who rated games before 10/10 just rate the games 5/5 instead. Basically just yielding the same result as before?

I guess people may have the same idea about Feillne's suggestion on a percentage based system but I do believe his suggestion is worth having a crack at.

Kyou. wrote: Wouldnt just changing the rating system from 1/10 to 1/5 just make the same people who rated games before 10/10 just rate the games 5/5 instead. Basically just yielding the same result as before?

I think having fewer numbers would encourage people to select from a wider range rather than just the extremes and as I've previously stated a star system would be even better (or so I believe). I just think the current system of having a huge buffet of numbers just makes people lost with what to choose.

Plus as I've already said it would lead to huge confusion, as people may interpret the developers intentions in multiple ways leading to completely bogus ratings which would actually make the whole situation worse! For example one person may interpret Minecraft's graphics as being totally rubbish because they don't like the large pixellated art style (despite this being intentional) and therefore would rate it very low. Whereas one person may understand that the style is intentional and therefore rate it highly (however in another situation they may have interpreted wrong). Put simply I think that form of rating system is too complex and would lead to way to many varying results which would make the job of deciding whether a game is good or not even harder than it is already.

Simplicity is key. 5 a star rating system is commonly used across many popular websites and works very effectively and I'd like to see this brought into Desura, IndieDB, ModDB and it's many other sister sites.

Graphics aren't the same as aesthetics. Minecraft does have poor graphics, but great aesthetics.

Here's an idea:

Reviews with no description gives the review significantly less weight.
The more people 'agree' with a review, the more the weight that review has.

Example:

Game gets a 10 star review from honest reviewer, who even puts down his thoughts.
Same game gets bombed with a 1 star, no description review.
Instead of averaging it at 5, the no description review is less affecting, so the review is raised to ~6 or ~7.

Second example:

Honest reviewer gives a nice review to a game with an average 7, but has 100 people 'agree'.
Same game gets bombed by ten 1's by various trolls.
The 7 outweighs the ten other reviews, and will likely end up around ~6-7.

Entirely aimed at shortening the 1s and 10s with no rhyme or reason, as well as promoting good reviews.

Edited by: JustDaveIsFine

Kyou. wrote: Wouldnt just changing the rating system from 1/10 to 1/5 just make the same people who rated games before 10/10 just rate the games 5/5 instead. Basically just yielding the same result as before?

I guess people may have the same idea about Feillne's suggestion on a percentage based system but I do believe his suggestion is worth having a crack at.

I would worry that people would just click "yes" on all questions in the questionaire style review, but its probaby the best suggestion there is to get people to really consider what they are putting down. Else we would have to go with a Like/Dislike kind of thing. This also gives kind of a warped look, but at least it could show how many likes and dislikes a game gets, and make a percentage. Then again, just because a game isn't a 10, it doesn't mean people dislike it.

ninjadave wrote: Graphics aren't the same as aesthetics. Minecraft does have poor graphics, but great aesthetics.

Here's an idea:

Reviews with no description gives the review significantly less weight.
The more people 'agree' with a review, the more the weight that review has.

Example:

Game gets a 10 star review from honest reviewer, who even puts down his thoughts.
Same game gets bombed with a 1 star, no description review.
Instead of averaging it at 5, the no description review is less affecting, so the review is raised to ~6 or ~7.

Second example:

Honest reviewer gives a nice review to a game with an average 7, but has 100 people 'agree'.
Same game gets bombed by ten 1's by various trolls.
The 7 outweighs the ten other reviews, and will likely end up around ~6-7.

Entirely aimed at shortening the 1s and 10s with no rhyme or reason, as well as promoting good reviews.

This is probably the best suggestion so far. I remember back on Newgrounds, they have this voting power, so if I have helped pass or fail x games, my voting power suddenly counts for 7 votes. So with your system, we could actually also incorporate a lot of other factors: Level on Desura could be a solid multiplier, because someone who has a level 10 surely has been around the block.

I would like to see this system implemented - heck, you could probably even use all the existing reviews, and just code the whole site to use this format.

ninjadave wrote: Second example:

Honest reviewer gives a nice review to a game with an average 7, but has 100 people 'agree'.
Same game gets bombed by ten 1's by various trolls.
The 7 outweighs the ten other reviews, and will likely end up around ~6-7.

Entirely aimed at shortening the 1s and 10s with no rhyme or reason, as well as promoting good reviews.

This is how it works already. The key here is we want people to write reviews and we want people to agree with existing reviews. My ideal review system is more of a recommendation system. Since games are naturally going to change over time reviews seem flawed to me.

Henley wrote:

ninjadave wrote: Second example:

Honest reviewer gives a nice review to a game with an average 7, but has 100 people 'agree'.
Same game gets bombed by ten 1's by various trolls.
The 7 outweighs the ten other reviews, and will likely end up around ~6-7.

Entirely aimed at shortening the 1s and 10s with no rhyme or reason, as well as promoting good reviews.

Maybe you could have reviews decay over time as well? Either when x weeks have passed, or when a game is updated. Decaying reviews also means that new games are put at the top.

The biggest problem is games that does not have written reviews, but all 10's. This give a very inaccurate view of how good the game is.

Perhaps, and call me outlandish, we could give all (new?) games 10x 5-ratings, and then really count on people to bring it up or down. Right now, I feel bad when I give a game a 7, even if its justified, because I'll be pulling its score down a bit.

This is how it works already. The key here is we want people to write reviews and we want people to agree with existing reviews. My ideal review system is more of a recommendation system. Since games are naturally going to change over time reviews seem flawed to me.

For me, there is a difference between the system and the people using it.

The 1-10 system is fine, but it's the mis-use/lack of understanding that kills it.

One example I remember is a review of Skyrim on Metacritic that went something like "I would give this game a 7 out of 10, so I rated it 0 to bring the average down to where it needs to be".

For me, I think the review system should be either a youtube style like/dislike system, or a 3 star system of "like, don't like, is ok/don't care". While not exactly something you could use for, say, a gaming website, it works in terms of anonymous/player ratings because most of the people who rate either hate the game, love the game, or are indifferent. Get rid of the middle ground entirely.

We could require 2 paragraphs of explanation to rate something, if you just fill it with garbage (like aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.............) then you can get reprimanded.