Bucks Diary

Sunday, November 09, 2008

The logic behind my RPI ratings and why the results are sometimes counterintuitive

There have been legitimate questions raised about my "Relative Performance Indicator" ratings I have been using to evaluate the game performance of the Milwaukee Bucks. Let me explain why the questions have arisen, why I went with the RPI method, and why I still believe it is the best method for evaluating the performance of each of the Milwaukee Bucks.

The questions have arisen (as I hoped they would... it means people are scrutinizing the information I provide) because of some queer results the RPI method has produced. (Click here to see the RPI results for last night's Bucks game). Specifically, last night Bucks center Andrew Bogut recorded what would ordinarily be a horrible Win Score 48 for a center: 8.5 (NBA center's normally average around 11.5-12.0). Yet, by RPI standards his performance was deemed "above average" (his RPI was a +0.349). The obvious response to that rating is... WTF??? He played like crap, didn't he?

The answer is: compared to the numbers other centers have produced against the Phoenix Suns this season, he actually played fairly well. To explain why that is, and why -- as strange as it is -- it has value, I have to briefly explain why I decided to go with a Relative Performance Rating in the first place.

At the beginning of the season I was comparing each Bucks performance against last season's averages. But doing some calculations, I noticed positional Win Scores in the NBA tend to fluctuate from season to season. Thus when in-season, I have no solid NBA-wide averages to reliably compare to the Bucks performances. And early in the season that discrepancy is magnified. The Bucks obviously play a limited mixture of Association teams until well into the scheule, so it really makes no sense to compare them against any existing Association-wide averages until the season's over.

So instead of doing that, I decided the best way to provide Bucks fans with a solid evaluation of how each Buck is doing at any given point in the season is to compare each Bucks performance against the performance turned in by the rest of the Association against the same opponent. Its the same basic thinking used by FootballOutsiders.com when they do their analysis of each NFL players game performance. Basically, they believe the best method for providing a neutral evaluation of the strength of any given player's performance is to ask how that player did compared to how the rest of the NFL did against the same opponent.

And I think it makes sense. Let's say you're John Hammond and you sign a guy to a ten day contract. And let's say within those 10 days you play some of the worst defenses in the Association. And lets further say the guy puts up huge stats against those weak opponents. After the 10 days expire, you have to decide whether to extend the guy. When doing so, are you going to take his 10 day stats at face value, or will you more likely adjust them to account for the lesser competition? I think it makes sense to adjust them. By doing so, you can project more accurately how the guy is likely to perform against the rest of the Association.

And that's all RPI does. It simply adjusts a player's statistical production in accordance with the production of the rest of the Association against that opponent.

So you're going to get some funny results early on. It just happened that the Phoenix Suns have been shutting opposing centers down to date. And it further happened that Bogut produced better statistics than the other centers have been able to produce... however weak those statistics may have been in absolute terms.

But I think, as weird a result as that was, it was still informative. Phoenix has been lockdown against centers. Bogut outproduced those other centers. Even though the sample size for Phoenix's defense has been small, doesn't Bogut's RPI suggest he may be able to outproduce other centers in other situations as well? After all, the Suns have probably faced about 12 centers prior to Bogut. Why couldn't those 12 produce the measly 8.6 WS48 Bogut produced? Was Bogut simply better?

That's all I'm trying to figure out. I'm trying to give Bucks fans an opponent-neutral evaluation of how each Buck is performing -- and how their defense is performing on each opponent -- on a game-by-game basis.


At November 9, 2008 at 9:47 PM, Anonymous paulpressey25 said...

Ty---humor me as I try to figure out all the different stats you use. What would Bogut's win contribution on offense and defense have looked like for last night's game? I was used to seeing those the past few weeks as you were calculating those scores before you shifted to RPI.

I realize the rationale that other centers have not done well against the Suns, but something has to be flawed in an analysis where a guy (Bogut) shoots 2-8 yet is considered to have an above average performance. Was it the 11 rebounds that brought his score up?

Also, with this new metric RPI, how does that effect how your counterpart plays? What I loved about your earlier scoring system was that it showed exactly how the guy you matched up against did. Can I find that type of analysis in the new RPI? I don't think I can. That defensive win contribution calculation appeared to be the genius piece of what you've been doing here.

At November 9, 2008 at 10:20 PM, Blogger TCW said...


Your killin me! I'm sensing you don't like my new approach. Maybe I'll go back.

The fact is I loved the direct counterpart statistics too, but the reason I ditched them was it was taking me too long after the games to put them together. I was staying up until 2 am just trying to accurately reconstruct the game.

I could do them the next day, but my concern was that people would consider it stale and uninteresting. But, maybe I should just do them in my free time, and when they're done they're done, and those that are interested can read them. What do you think about that?

Oh, there's one other thing. The second reason I decided not to do the direct boxscore was I was noticing my numbers didn't jibe precisely with 82games.com. And in the long run, I have to rely on them for "counterpart" information. And sooner or later someone would notice that my game statistics didn't square with my weekly "Win Contribution" charts.

I don't know, I have to think about this. I gotta give the people what they want. Maybe I'll just do boxscores when I can and just post them up as sort of an FYI for Bucks fans.

At November 10, 2008 at 4:41 AM, Anonymous Steve from Aus said...

I think it's an interesting approach to use, but it's obviously necessary to step back and look at it with some common sense. Bogut was garbage last night. The fact that other centers have also been garbage against the suns shouldn't matter.

If you could find a way to link your two methods it could be great- ie have a model that integrates the RPI as well as the winscore- winscore can be adjusted to a degree based on the RPI or whatever. Probably not possible to do but it would result in more defensible and practical looking stats.


Post a Comment

Links to this post:

Create a Link

<< Home