Monday, July 26, 2004

ESPN NFL2K5 Slider Project: The Results

I bet you weren't expecting to see that headline for a few days.

Here's what happened. The combination of a limited number of sliders to adjust and a high number of volunteers made the analysis go very quickly. We're done.

There are caveats to 'done.' The core CPU vs. CPU game is tilted, to some degree, toward defense. So there were logical adjustments to make, which reduced the tilt but did not completely eliminate it. Without being able to reach what I considered balance, that eliminated the fine-tuning phase. It was more a process of 'get it as close as you can.'

Not being able to totally balance CPU vs. CPU games is not necessarily relevant to our final experience with the game. The slider settings aren't used for simulated games, as far as I can tell. Balancing CPU vs. CPU games is more a process to help ultimately balance CPU vs. Human games than an end in itself.

Here is what we learned, and mostly these are balance limitations, not praise, although I'll have plenty of praise later in the week when I have a discussion of the full game.
1. CPU quarterbacks completion percentage is not consistently high enough. Along with this, they throw too many interceptions in CPU vs. CPU games. I've noticed that they throw into heavy coverage too often. The slider settings minimize this as much as possible, and I will say that I'm not seeing problems with quarterback efficiency when they're playing against me, because I'm generally getting my brains beat in (using the Chargers).
2. With only one slider to adjust kicking, compromises have to be made. Even with kicking slider reduced to zero, kickoffs are still a little too strong, while punting is a little too weak. Plus each kicker is far too consistent in terms of distance when kicking off. On the plus side, field goals are missed pretty realistically, which fixes a huge problem from last year.
3. Not enough penalties are called. Even with almost all of the penalty sliders maxed, a regular number of penalties called during a game is only five or six. Plus I almost never see false starts.

The most interesting outcome of the statistics people were sending me from simmed games were the outliers. And instead of paraphrasing, here's an exact definition: Outliers are the observations that appear to be inconsistent with the reminder of the collected data (Iglewicz, 1993, and thanks to Using the slider settings which produce, by far, the best median statistical results in CPU vs. CPU games will also produce more outliers than I expect to see. Players will just have absolutely horrible games more often than you'd expect.

So there is more variation possible inside the game engine than I am used to seeing with Madden, for example.

Some of that sounds fairly damning, for those of you who use the word 'damning' when referring to the statistical outcomes of CPU vs. CPU games in football simulations.

You think I have 'damning' taped to my monitor, don't you? I do, and it's typed in CAPS.

In spite of being the hard-ass that I am about sports games, though, and even though these are issues that I'd normally get worked up about, this year I just can't. The in-game experience on a second-to-second basis is generally so overwhelmingly positive that ESPN is a joy to play. I don't use that word often, that's for damn sure, but it applies here. It is absolutely wonderful, even with warts. I'm going to expand on this thought later this week, but the easiest way to compare ESPN to Madden is to say that ESPN has aspects that are so soaring, so brilliant, that they absolutely demolish Madden, but that Madden is so workmanlike and steady that when ESPN falters it is always there with something better. That's why reviewers frequently have such a difficult time comparing these games and why they disagree so strongly about their merits.

All right, enough blabbing. Here are the slider settings. Quarter length for played games should correspond to whatever gives you 110-120 plays a game (I use seven or eight minutes). Oh, and it appears that setting simulated games to use 6 minute quarters will produce the best Simulated stats). Sliders are on a 1-40 scale, with 1 being min and 40 being max, and there are tick marks every 8 places on the slider scale (I stuck to increments of 8 to make them easier to adjust). I use the same setting for both CPU and Human.

Blocking 40
Passing 40
Running 40
Catching 40
Coverage 0
Pursuit 8
Tackling 8
Kicking 0
Fatigue 20
Injury 20
Fumbles 32
Interceptions 0

We didn't test the Fatigue or Injury settings. Fatigue is another project in itself, and some of those are already underway by people who will do it better than I can.

Now if you look at these settings and think that I've eliminated drops by maxing out catching, for example, that's not how they work. The min/max for the slider isn't an absolute--it's a relative influence, and the max/min influences do not produce absolute results.

For Penalties, all sliders are maxed except clipping, roughing the kicker, and ineligible man downfield, which should be set to 32.

That's it. I've played several excellent games with these, and I hope your experience is the same.

Site Meter