Carleton: How reliable are our fielding metrics?
From Russell Carleton at Baseball Prospectus on May 14, 2013:
A little more than a week ago, Jon Heyman of CBS sent out a tweet wondering why it was that Starling Marte and Bryce Harper had the same WAR. Heyman was quoting Baseball-Reference’s version of WAR, which at that moment in time showed Marte and Harper tied at 1.7 wins. Harper had clearly been the superior hitter, but drilling down, it turned out that the fielding metric used by Baseball-Reference loved Marte’s defense enough (and thought Harper’s was average enough) to call them equals.
The problem with any sort of number this early in the season is that on many measurements, we’re still at a time when players haven’t logged enough playing time for the measure to be considered reliable. But of course, some measures are more reliable than others. The more reliable a measure, the sooner we can be more confident that it actually reflects what the player’s talent level was during that time. The less reliable it is, the more likely it is that there will be fluky spikes and valleys over short (and sometimes long) periods of time. Fielding metrics are an estimate of how many outs a player saved from Opening Day onward, and what that was worth. However, in the same way that a player who went 3-for-4 on Opening Day is technically a .750 hitter for the moment, it’s not real. A fielding metric might need some time to stabilize as well before we get a good read on what’s going on.
There’s been research on how quickly various batting and pitching statistics stabilize, but in general, few people have asked the question of how reliable our fielding metrics are. One reason is that several of the most commonly cited fielding metrics (UZR, DRS) rely on proprietary data not available to the general public. We just don’t have the ability to peek under the hood. And so, we’re going to have to get a little creative.
Read the full article here: http://www.baseballprospectus.com/article.php?articleid=20571
Originally published: May 14, 2013. Last Updated: May 14, 2013.