If you pay any attention whatsoever to gaming reviews, surely you are familiar with Metacritic, a site which presents aggregate scores for games, movies and more based on the reviews of many reliable websites. Recently they have had a bit of trouble with user scores for popular titles such as LittleBigPlanet and Resistance 2, with user scores rating far lower than the Metascore — so much so that it seemed people were intentionally abusing the privilege of being able to contribute their own reviews.
Our own Jim Sterling has given us his take on the subject, and while it makes some excellent points, you certainly can’t beat sitting down with the source. I had a chat with Metacritic’s Marc Doyle today, one of the co-founders of the site, to get right down to the bottom of the havoc created by tireless fanboys and trolls.
DESTRUCTOID: Some members of the industry have said they would like to see Metacritic drop user scores altogether and only publish an aggregate of press scores. How do you respond to those comments?
MARC DOYLE: I’m surprised to hear that actually, because the feedback I get is that the industry likes to see user comments. Of course, some people will try to manipulate the system, which has happened previously on a a few instances, such as over the first The Lord of The Rings film. Being entertainment junkies ourselves, we have always wanted to allow people to vote early, but the recent events have caused us to remove the option.
DESTRUCTOID: Why do you think the issue with unreasonably low user scores is occurring more on Metacritic than on other sites?
MARC DOYLE: It’s certainly due to the ease of our sign-up process. Our three co-founders were all lawyers before creating Metacritic, and people’s privacy was a definite concern in our vision of the site. We wanted to make the process very simple for people to be able to comment, which is likely also lending to these low user scores — if fans of one console want to rate down a version of a game on another console, they could do so with ease.
DESTRUCTOID: Unfortunately, it seems people that used the site prudently will now have to pay a price because of these people who abused the privilege.
MARC DOYLE: That’s right. However, we are in the midst of planning a redesign which will feature a robust user registration system. It will work the same way other site networks do — for instance, by signing up for CNET, you would also be able to use Metacritic using the same log in. I can’t say when this redesign will go into effect, but it is definitely on the horizon.
DESTRUCTOID: Do you feel a stricter sign-up process will really fix this issue, considering that trolls are notorious for their persistence? Would Metacritic consider banning users if they continue to abuse the site even after the redesign?
MARC DOYLE: We will consider it if it is necessary. Obviously we don’t want to take away a user’s freedom to post reviews on the site, but a person who is intentionally trying to sabotage a game’s score will certainly not be ignored. We want the site to be a pleasant experience for users, and we will do what we have to do to preserve that.
DESTRUCTOID: In your opinion, do Metacritic users pay more attention to the user reviews, the professional reviews, or a mix of both?
MARC DOYLE: Well, the Metascore is considered our primary product, and that is generated from the reviews of publications we have hand-picked to contribute to it. The intent of the founders was to allow users to post reviews to support the product along with the Metascore review, but obviously it’s the Metascore itself that we place focus on. We think of the user reviews like a tally of votes, it helps us to keep track of how many other opinions are out there.
DESTRUCTOID: Have you been contacted by publishers in regards to any of the famously low user scores for major games like LittleBigPlanet and Gears of War 2?
MARC DOYLE: I haven’t when it comes to user scores, no. I think the publishers know the nature of the beast, so they don’t question the user scores. I get calls about a lot of the other things that go up on the site, but luckily user reviews aren’t one of them.
[Destructoid would like to thank Mr. Doyle for his time.]
Published: Nov 13, 2008 03:04 pm