Game Rankings
Tom Chick wrote a column over at Quarter to Three about game rankings, and like anything he writes, it's interesting. Here's a link (thanks Steve Smith):http://www.quartertothree.com/inhouse/news/246/.
Here are a few excerpts from that column:
The cat is out of the bag that Computer Gaming World is dropping review scores in favor of longer and more in-depth articles. And, predictably, there is a wide range of reactions.
...I have to wonder why you're so enamored of your ratings. I can understand a guy like Stardock's Brad Wardell, who's concerned about the commercial implications of the decision: when a respected voice like CGW is removed from sites that aggregate score, then sites that aggregate scores will be less meaningful. To which I say, it's about *&%$ing time.
I'm all for aggregate sites if that's how you want to make your decisions. When you go to the multiplex this weekend, let Rotten Tomatoes do the thinking for you. Have fun at Eight Below. I'm sure it's heart-warming.
So if Gamerankings is how you choose the games you buy, then you deserve Perfect Dark (82%) over Kameo (81%). Black (78.3%) should be just your speed, while the really shrewd America's Army: Rise of Soldier (69.5%) will likely be lost on you. To people like you, there's no point trying something utterly genius like Fatal Frame III (80%) when you could be playing Resident Evil 4 (96.1%), which is nearly one fifth better. Personally, I couldn't care less if you miss out on the clever and sharply designed Eve of Destruction (73.5%) because you're busy with glib pap like Burnout 3 (93.6%). I don't recommend you bother with the unexpected gems I've found like Chibi-Robo (75%), Romancing Saga (64%), Killer 7 (77%), or Romance of the Three Kingdoms X (70.2%).
Math won't help you understand these things, which is a fact lost on so many of the people who write about games, many of whom don't even have the basest appreciation for what words mean, much less how to use them well. Math is easy. It's words that are hard.
And what hurts the industry more than whether CGW’s review is figured into Gamerankings.com is the proliferation of the same dull poorly-written-by-template coverage...So if you have a love of this hobby that calls for a bit more analysis, creative consideration, and articulation than it's currently getting -- which is to say next to none at all -- then the first step is to force guys like me who write about games to step away from the 7.9s vs 8.1s, away from the non-critical dull review formats that drive what passes for game reviewing, and into something different that requires, you know, actually making, like, a point?
I bolded what I'm about to discuss--the notion that game ranking aggregation is shit and a bad way to make decisions about what games you want to play. He says much more, and it's well worth reading, so I encourage you to read the full column and come back. It's very difficult to excerpt Tom's writing, because it's wound very tightly together.If you've read this column for any length of time, you know that I like Tom Chick's writing.
I wrote a column back in July that analyzed the content of all the major gaming magazines
(http://dubiousquality.blogspot.com/2005/07/gaming-journalism-in-print.html), looking for unique content, and here was one of my conclusions:
Here’s a funny stat: Tom Chick is responsible for over 15% of the unique content of THREE PC gaming magazines COMBINED.
If there's ever a Hall of Fame for game reviewers (and believe me, some dumbass will create one), Tom Chick should probably be the first inductee. So understand my respect when I say that, in this case, he's wrong.
Look. Taking anything and finding an imperfection is meaningless. Chick taking games that are percentage points apart and mocking an aggregate score comparison is a straw man. The question is not whether game ranking aggregates are perfect--obviously, they're not--but whether they are useful.
And it's worth mentioning that no single reviewer is perfect, either. Mark Asher (another writer whose work I really enjoy) picked Flying Heroes as the best game of 2000. No one else on this planet thought that Flying Heroes was the best game of 2000. That doesn't mean Mark Asher isn't an excellent writer and reviewer (he is). It just means that no person or system is perfect. They can't be.
Here's why I like aggregate review sites like Gamerankings.com or Metacritic or Rotten Tomatoes: everyone's bias is represented. If every single reviewer has bias and a limited perspective, and if I see fifty reviews, I know that it represents a far more comprehensive perspective.
And, to a large degree, it works.
Let's look at a cross-section of rankings on Metacritic. Here are their top-ten ranked PC games of all time:
--Half-Life 2
--Half-Life
--Baldur's Gate II: Shadows of Amn
--Command & Conquer
--Grand Theft Auto: Vice City
--Civilization II
--Quake
--Grim Fandango
--Diablo
--Civilization IV
That's in order. Civ IV has a 93--the others are higher. And it appears that their database goes back to 1993, so games before then are excluded.
Are those great games? Certainly, most of them are. I'm not a huge fan of GTA, but hell, almost everyone else is, so that seems fair.
Now let's take the first ten games that have an overall rank of 85.
--Tom Clancy's Rainbow Six
--Falcon 4.0
--Madden NFL 2005
--Everquest
--Clive Barker's Undying
--City of Heroes
--Operation Flashpoint: Cold War Crisis
--Aliens vs. Predators 2
--Giants: Citizen Kabuto
--Tropico
Tropico's one of my all-time favorite games, so I think it got jobbed, but everything else looks pretty solid. Falcon 4.0, remember, was released with a few million bugs, and that score reflects a ton of reviews at release or shortly thereafter.
Let's try 75 (first ten games).
--Project: Snowblind
--Tropico 2: Pirate Cove
--Space Colony
--Patrician III
--Command and Conquer Renegade
--Port Royale 2
--Total Pro Football 2004
--Blitzkrieg 2
--MTX: Mototrax
--Sims: The Vacation
I wouldn't put anything from this group above anything in the second group. And I definitely wouldn't put anything from this group in the first group.
Here's 65 (first ten games):
--Stupid Invaders
--Battle Engine Aquila
--RC Cars
--Battleship: Surface Thunder
--Casino Empire
--Future Boy!
--CSI: Dark Motives
--X-Com Enforcer
--Journey to the Center of the Earth
--US Open 2002
Getting pretty skanky down here. And these games are generally worse than the 75 games, and they're definitely all worse than the 85 games.
Oh, no. Here's 55:
--I of the Dragon
--Casino Inc.
--Aliens vs. Predator 2: Primal Hunt
--Suzuki Alstare Extreme Racing
--Emergency 2: The Ultimate Fight for Life
--X2: Wolverine's Revenge
--Shrek 2
--The Incredibles
--House of the Dead III
--Codename: Outbreak
And one last time, for 45:
--Heaven & Hell
--Splat Magazine Renegade Paintball
--Supercar Street Challenge
--Big Mutha Truckers 2: Truck Me Harder
--World War II: Panzer Claws II
--Cops 2170: The Power of Law
--Dragonriders: Chronicles of Pern
--Salt Lake 2002
--Empire of Magic
--Bad Boys II: Miami Takedown
That data's pretty clear. Aggregate scores generally work.
Is Metacritic's aggregate score system perfect? No. Is it highly useful? Clearly, yes.
I mean, come on. Every game review should answer one question: Is it fun to play and for how long? It's not rocket science. That some writers have done much more with the form doesn't mean that it's essential to the form.
There's another reason to use aggregate game rankings as a factor in purchase decisions: the number of games. If Tom reviews five games a month, that would be a tremendous output. Meanwhile, a hundred games are coming out in March. Yes, I'm counting multiple platform titles on each platform they're releasing in March, but even removing those, the total number of games across all platforms is staggering. So even if I wanted to rely on the best reviewers and their nuanced, thoughtful impressions, they're not going to be able to review most of the games.
It's irrelevant, in the end, if Computer Gaming World includes rankings in their reviews or not. That's true of any print magazine, because at this point, all their reviews are irrelevant. The delay in publishing a print magazine is a Catch-22. If they have a timely review, it's because they reviewed beta code (hello, PC Gamer "exclusives"), and it's worthless. If they review final code, we can't read the review until two months (or more) after the game is released. At that point, twenty or fifty or a hundred online reviews have been written (some by major gaming sites, many by minor) and published. Is one extremely well-written print review more useful?
No.
And removing rankings from game reviews isn't going to make a bad writer a good one. There's a very small tier of people with excellent writing and analytical skills; below them, there's a giant sea of mediocre.
The best we can hope for is that a few people emerge from the sea, and I think that's fine. Do I wish game reviews were better written and more thoughtful? Absolutely. Am I going to lose any sleep over it?
Absolutely not.
<< Home