Tuesday, July 21, 2009

NCAA 10 (360): A Brown, Gelatinous Mass That Is Only Vaguely Shaped Like A Football

As a fair warning, I thought NCAA was a complete disaster last year, a bad early beta that cost us $60. Previous to last year's version, though, even with its mounting number of flaws, I'd always felt that I'd found a way to get my money's worth.

When the game shipped this year with rosters fouled up and the sliders not working (I had to download "fixed" rosters, and a patch for the sliders is supposed to come out tonight for the 360 version), it reinforced my notion that this game was going to be a lazy, buggy mess.

Reading the "official" bug thread over at Operation Sports reinforces that impression (with big-time issues with fatigue and, incredibly, spotting a punt correctly when it goes out of bounds, plus loads of others as well), but this is a monopoly, we don't have any other choices, and I'm still hopeful that patches will fix enough of this to be able to enjoy the game.

I decided to test Dynasty mode in terms of career progression (since, without the sliders working, I'm not playing any games yet), which has been a significant issue in the past for this series.

In short, this is how progression should work. For the game to play consistently over time, player ratings (as a composite) should basically stay the same every year. Individual ratings change, obviously, but if you have 10 receivers with an overall rating of 95+ in the first year of a dynasty, that number should be maintained to a reasonable degree by the player progression system.

How should that be tested on the development end? Well, the dev runs a Dynasty for 5-10 years and dumps the player ratings at the end of each year into a text file, then assigns "bins" by position and rating. For quarterbacks, for example, there would be a bin every 5 ratings points or so, and the testing tool would tally up how many quarterbacks in the game (for example) have an overall rating between 70-74. There would be a total for every 5-point range (there's an easy way to do this in Excel).

Once the create player algorithm has run for enough years that the default rosters are no longer involved, that final output is the steady state for the progression algorithm. And if it it changes significantly over a five-year cycle, then the progression algorithm has to be adjusted.

Once that's stable, the dev takes the bin output and checks the default player ratings (that are created manually, not by the player creation algorithm). If the bins for the default ratings are a mismatch with the steady state bins, then the default ratings have to be adjusted until the bins are a close match.

That could theoretically be done for every skill rating in the game as well.

What's most important to keep in mind is stability. These ratings need to be stable over time, because if they're not, the way the game plays is going to change, and stats will change substantially.

If that sounds complicated, it's really not. It's just precise work, and it's repetitive. It takes many iterations to get it correct (I was responsible for this kind of testing in three baseball games over the years--twice as a beta volunteer, and once on my own after a game had shipped--modifying the player creation algorithm to create stable statistics over a 20-year period).

So this is one of those areas where I can claim expertise well beyond the level of "I don't make chairs, but this leg shouldn't be wobbly." I've very aware of the fact this this is a detailed, time-consuming process.

So how does NCAA 10 do in this area? Here's a snapshot:


Please click on the image for a larger version.

The summarized version: there were 14 players in the entire game (any position) with an overall rating of 95 or higher on the default rosters. After five years, there were 75! Also, there were an additional 112 players with overall ratings of 90-94, and after five years, there were 270.

That's kind of a shitty mess, really, and some positions are just mind-boggling. There were originally two centers at 95+ and 8 more from 90-94, but after five years, there were 18 at 95+ and 36 from 90-94!

Middle linebackers and cornerbacks are a disaster as well. Actually, there are quite a few disasters. It's a very poor job.

I did notice something that is oddly enticing. I listed the positions in the same order that they're listed on the roster screen, and if you look more closely, you'll see that the first six positions look very good after five years. Yes, there's some hyper-inflation in the middle, but after a full five-year class has been run through, everything is quite good, actually.

Further down the list, though, it's generally a train wreck. Given the the positions that are stable are the first six listed, did they just run out of time?

I've said this before, but I believe it even more strongly this year: the NCAA franchise is entirely lacking in leadership. There is absolutely no excuse for this game to come out in the shape it comes out in year after year, but no one at EA is ever willing to accept responsibility.

Because I'm Charlie Brown, though, I'll keep trying. I'll try to balance the on-field play with slider adjustments (when they finally work). I'll try out legend mode (even though the overtime bug that made me quit last year has apparently not been **$*#@ patched). We're screwed on player progression, though, unless EA decides to include it in a patch.

I'll let you know what happens.

Site Meter