The Limits Of Sabermetric Accuracy (Ruminations from the Southeast Tower)

It’s been an unprofitable year here at Fletcher Estates. The orchards have been a little unproductive, the trucking company lost a bit of money (but experienced significant growth; the future there, at least, is promising), and I’ve been a bit too lacking in energy to bother about replacing my wardrobe this year.

My wife is talking about us flying to Florida and boarding another cruise. She loves to travel and god knows I owe her everything, but still… Couldn’t I talk her into a major league ballpark tour and…well, no.

No, I’ll just satisfy my selfish nature the usual way and retreat to the southeast tower of the castle, the one that overlooks the apple trees, the blueberry farm and the riding stables, the one with the air conditioned office where I compose my witty posts to Bill James Online and give my attention to things sabermetric, somewhat analytical, though mostly philosophical.

I think to myself, consider the wonderful simplicity of counting statistics. Take this players career numbers, for example:

G         PA       AB      R         H         2B       3B       HR

1403    5442    5094    731      1397    227      71        67

The nice thing about these basic numbers is that they represent things that really and truly did happen. Take the number of doubles. Two hundred and twenty-seven times Mookie Wilson came to the plate and hit a fair ball and ended up safely on second base. Maybe he would have done so more often playing in different home parks than he did, or maybe he would have done so less often. But this is what really happened.

Is this a good total of doubles for an outfielder? Playing in the National League from 1980 to mid-summer 1989, and in the American League from mid-summer 1989 to 1991? Playing at home in Shea Stadium in the first case and in the newly opened Skydome in the second? Is it a good total per PA or per AB?

I don’t care.

I am just noting something very obvious, something that I believe we all know, that the raw data represent unadorned, simple facts. We all know this, but we want more. We want to be able to say, for example, that some player led the league in Home Runs, therefore he is the best Home Run hitter in the league. We’d like to, but wait a minute… We ask ourselves, is that really true? What about this player who hit just 2 fewer Home Runs but did so in 123 fewer plate appearances? What about this player who plays in AT&T Park and hit 7 fewer Home Runs in about the same number of plate appearances? And these are only the most obvious and coarse ways of looking at it, just the start of our desire to move away from the rock hard facts and move toward assessing value, of judging ability, or projecting future production…and maybe even of asking ourselves just what it is we are asking.

Okay, here is what I am thinking. I think that all of our attempts to derive real meaning from baseball statistics do so at the cost of distorting objective reality. I believe (not know…believe) that the more adjustments we make in our attempts to find meaningful truths, the more the potential distortion.

(It is also true that the raw statistics need the aid of telescopes and microscopes to extract qualitative answers, as in that effort we need to change the focus. But still, we start with what happened, not what could happen.)


I remember when I first saw the Runs Created formula. Here it is:




Beautiful, elegant thing, isn’t it? As its inventor noted the formula can be simplified even\ further this way:




But, perhaps anticipating the objection (or more likely having his own concerns), Bill James worked in Stolen Bases and Caught Stealing:




This version has the virtue of being more complicated but not particularly more accurate. But of course if stolen bases and caught stealing weren’t identified in the formula there would have been all sorts of objections.

It would be nice if we could leave it right there, but of course we couldn’t, and we shouldn’t, and we didn’t. (The “we” here being the sabermetric community.) The runs Created formula (and there are many and more complicated versions if you like) was and is wonderfully helpful in taking all those individual counting statistics (singles, doubles, triples, home runs, walks, stolen bases, caught stealing…and hit by pitch, and grounded into double plays, and at bats or plate appearances) and bringing us a single number from which we could advance our efforts to understand.

Well, I hope you are excusing me for this trip down memory lane. Time has passed and we’ve all moved on. We have way more information being tracked now and way more methods to analyze it all. We have BAbip, we have WAR, and her children oWAR and dWAR. We have WHIP and FIP. One BJOL member (who I would like to credit but I can’t remember who) noted that you could insert a bunch of these acronyms into a Don Martin comic strip. (Just for fun: BORFFT, FAGROON, GASHOOK, SHKLIK).

You might think I am preparing to criticize the hell out of these methods, but actually I think they are quite beautiful, if a bit ornate. The logic of these estimates, though often difficult and somewhat torturous, is quite reasonable. And as long as these things are reasonable I am glad to let them speak to me.

I believe in the uncertainty principle. Not the physics so much (which I trust but don’t particularly understand) but in the philosophical rendering, which is that the very act of measurement changes what is being measured. I absolutely believe that the simple act of counting the common events of baseball games does this.

Granting the evident truth of that last sentence I think it follows that every time we multiply, divide, chart, convert, extrapolate, regress…you name it…we add just a little bit to the uncertainty.

The evening approaches and I think it is time to re-introduce myself to my wife. I want to end the days musings by saying that I really appreciate all of these measures, but also want to express my appreciation that Killebrew may not always = McCovey, in fact Killebrew is almost certainly either > than McCovey or < McCovey. There will always be room for doubt and opinion.


GF – September 1, 2015


Leave a reply