1UP And A Curious Inflation
On July 10, I wrote a post titled The Great Horn Of Bullshit, in which I discussed this claim of Electronic Arts CEO John Riccitiello:All Metacritics were higher once upon a time because it was ten professionals rating them. Now, sort of anybody with a pen can rate them and it ends up with a bit of a wider track some times.
Based on a comparison of review averages from four "professional" sites (IGN, Gamespot, EGM, and Gamespy) for the major team franchises of EA Sports from 2000-2008, there was no negative bias from "anybody with pen" compared to the "professionals." I also didn't find any positive bias from the "professionals."
I forgot, though, about 1UP.
In the original post, I intended to use 1UP as the fourth "professional source," but the review data didn't go back nearly as far as the other sites I chose. Today, though, I saw their review of Tiger Woods 09-- an "A."
Hmm. Madden got an "A" as well. NCAA got an "A-." Do they just give every EA Sports game an "A"?
No, not exactly, but the padding above the average Metacritic score is pretty remarkable. Let's take a look at the letter grade given in the review, as well as the variance from the average Metacritic score. Please note that in converting the letter grades, I used this scale: "+" is an 8 (98=A+, 88=B+, etc.), no modifier is a 5 (95=A, 85=B, etc.), and "-" is a 2 (92=A-, 82=B-, etc.). Also please note that Gamerankings (data source) often has a score listed in the "all reviews" section that doesn't match the letter grade given in the actual review (like listing an "85" when the game was given an "A-" in the actual review). Because of that, I looked at each review individually to verify the letter grade.
2008:
Madden NFL 09=A (+9, 2nd highest of 26 reviews)
NCAA Football 09=A- (+7, 3rd highest of 20 reviews)
Tiger Woods 09=A (not enough reviews yet)
UEFA Euro 2008=B+ (+8, 3rd highest of 39 reviews)
2007:
Fifa Soccer 08=A (+12, highest of 47 reviews)
Madden NFL 08=A (+13, highest of 51 reviews)
NBA Live 08=C+ (+5, 15th highest of 39 reviews)
NCAA Football 08=A- (+10, highest of 31 reviews)
NCAA March Madness 07=C+ (+8, 7th highest of 30 reviews)
NCAA March Madness 08=C+ (+8, 4th highest of 22 reviews)
NHL 08=A (+10, 2nd highest of 43 reviews)
Tiger Woods 08=B (+6, 11th highest of 49 reviews)
UEFA Champ League 2006-2007=C+ (+4, 13th highest of 29 reviews)
Wow. That's 8.5 points higher, on average, than the Metacritic average. And the average review was in the 15th percentile of all reviews.
I started off with EA Sports games, but 1UP score inflation is a nearly equal opportunity employer. Team sports games from other publishers also fared well in the last two years, although the variance was less-- only +5 points, on average.
Average variance for the 20 games I tabulated?
1UP +7.45
EGM -2.35
IGN +2.37
Gamespot -2.13
If you're wondering which site had the least positive or negative variance from the average review score, the hands-down winner was Gamespot. And if you're wondering why Gamespy isn't included, it's because I completely forgot. That happens when you get old.
For some reason, until I compiled this data, I actually thought that 1UP had the most accurate sports reviews, but clearly, that was wrong. Big wrong.
<< Home