Wednesday, March 05, 2008

Hen's Teeth and the 85+ Review Rating

Matt Matthews did another excellent piece (I sound like a broken record) over at Next-Gen titled The Ratings Rankings. Using GameRankings, he came up with an average review score by publisher, for each of the last three years.

I highly recommend reading Matt's article, but I wanted to look at the same data from a slightly different angle, because as a consumer, the average review rating for a publisher doesn't affect me much. I'm much more interested in the highly rated games, because that's going to be the vast majority of what I play.

I used Metacritic (easier to cut and paste into Excel), and here are a few interesting numbers. Oh, and as a caveat, I had to make some decisions in terms of how to count the same game being released on multiple platforms. Since I saw quite a bit of score variation between platforms (much to my surprise), I'm counting each release on each platform as one game.

Beginning in 2006, there are 1070 games listed in the Metacritic database for the 360, Wii, PS3, and PC. Take a look at the review distribution (with >85 generally a "very good" game and <70>
That distribution was somewhat shocking to me. I knew that bad games were more common than very good ones, but seven times more likely? Yikes. And only one game in fourteen getting average review scores over 85?

Here's a breakdown by publisher. Again, one game can count more than once if it comes out on different platforms, but I think they should be rewarded if they maintain the same high quality across platforms. Oh, and since the PS2 and Gamecube aren't included, this is skewed against Sony and Nintendo, since their new platforms came out a year later than Microsoft's (although since I don't start this until 2006, so the 360 launch games aren't included).

9--EA (Rock Band, Burnout Paradise, Burnout Revenge, Skate, NHL 08, Crysis, Fight Night Round 3)
8--Take-Two (Oblivion + expansions, BioShock)
7--Valve (Half-Life2: Episode One, Portal, Team Fortress 2, Orange Box)
6--Microsoft (Gears of War, Forza 2, Mass Effect, Halo 3, Rez HD)
5--Ubisoft (GRAW, GRAW2, Rainbox Six Vegas, IL-2 Sturmovik: 1946)
4--THQ (Company of Heroes + expansion, DOW expansion, Supreme Commander)
4--Sega (FM 2007 and 2008, Medieval II: Total War, Virtua Fighter V)
4--Nintendo (Super Mario Galaxy, Super Smash Brothers Brawl, Legend of Zelda: Twilight Princess, Metroid Prime 3: Corruption)
3--Stardock (Gal Civ expansions, Sins of a Solar Empire)
3--Sony (Resistance, Ratchet & Clank, Uncharted)

The numbers don't always match the number of games because of multiple platform releases.

That's it. No other publisher had three titles during that time period with average review ratings over 85. I didn't realize until I compiled this list that for any publisher, three 85+ games a year is generally the high end for quality. And if you don't count multiple platform releases more than once, you wind up with about forty games in the last three years, or just over one game a month.

I think there are two publishers who stand out here: Valve and Stardock. Their hit-t0-miss ratio has been very high in the last two years compared to anyone else. Yes, Steam makes it very confusing to calculate numbers for Valve, but anything they develop themselves and publish is gold.

There are a ton of ways to slice this data, and I'm sure the results would look different if I chose 80+ instead of 85+ as the cutoff point. It's just that every game I saw on that 85+ list that I've played, even if it wasn't my kind of game, had a degree of quality that I wouldn't dispute. In the 80-85 range, though, I didn't feel that way.

This won't be the only time I look at this data. Now that it's in Excel, I can view it from all kinds of different angles.

Site Meter