Great Streaks: A Response to Trent McCotter

In an article in the 2008 issue of The Baseball Research Journal, Trent McCotter argued that hitting streaks are achieved more frequently than if there were no “hot hand” effect.1 Here, the author acknowledges that finding, but argues that the effect is so small that it can be ignored for practical purposes. In addition, he uses the original researcher’s technique to identify the most seemingly-unlikely (although not necessarily longest) streaks of the past several baseball seasons.

Introduction

Recently for the Journal of Quantitative Analysis in Sports I wrote an article where I looked carefully at the streaky patterns of hitters during the 2005 season.2 After I wrote this, I vowed never again to write about streakiness. But after reading the recent article by Trent McCotter in The Baseball Research Journal, I had to break my vow. McCotter’s interesting look at streaky hitting and the statements made in the article deserve some explanation and comments. Also, he describes an attractive method for assessing streakiness and it is straightforward to apply his statistical approach to identify extreme hitting streaks in recent seasons. Using this methodology, I find some “great streaks” during the 2004 through 2008 baseball seasons. 

Comments on BRJ Article by McCotter

In the BRJ article, McCotter wishes to construct a test of the common hypothesis that the individual batting outcomes of a particular player during a season represent independent, identically distributed trials. (We’ll call this the “IID assumption” or the “IID model.”) Essentially this hypothesis says that the batting outcomes are similar to flips of a coin where the chance of a hit on a single at-bat is equal to the batter’s “true” batting average. 

To test this hypothesis, the author looks at the pattern of game hitting streaks of all players for the seasons 1957 through 2006. Suppose we collect the game-to-game hitting records of Mickey Mantle during the 1961 season. We record all his hitting streaks for this season—maybe he started with a hitting streak of one game, a second hitting streak of three games, a third hitting streak of four games, and so on. If the IID assumption is true, then the pattern of hitting streaks for Mantle is simply a by-product of chance variation. If we randomly rearranged his game-to-game hitting statistics, then that wouldn’t change the pattern of streaks. The general question is whether Mickey Mantle’s observed pattern of hitting streaks (and the streak patterns for other players) is consistent with the random patterns from a model with the IID assumption. 

The author performs a computationally-intensive simulation of the patterns of hitting streaks for all players and seasons for the period 1957–2006. For each player’s game log for a season (for all 50 seasons), he randomly arranges the batting lines. Then he computes the lengths of all hitting streaks for all players for all seasons. Then he repeats this simulation process for a total of 10,000 iterations. When he is done, one has an empirical distribution of the lengths of hitting streaks under the IID assumption, and one can see if the actual lengths of streaks are consistent with this distribution. 

The conclusions of the paper can be summarized by two tables that show that the actual number of long hitting streaks (of length 5 and greater) are consistently larger than the mean number of long streaks predicted from the IID model. Moreover, the differences are highly statistically significant. The author concludes by saying: 

This study seems to provide some strong evidence that players’ games are not independent, identically distributed trials, as statisticians have assumed all these years, and it may even provide evidence that things like hot hands are a part of baseball streaks. . . . 

From the overwhelming evidence of the permutations, it appears that, when the same math formulas used for coin tosses are used for hitting streaks, the probabilities they yield are incorrect. 

Much of the article is devoted to a discussion of this conclusion, giving some possible explanations for the presence of long streaks.

Generally I’m fine with the statistical methodology used in the paper. As I’ll illustrate later, the permutation test procedure is an attractive method for testing the IID assumption and the results described in the article are interesting. But I am concerned about some of the author’s statements and conclusions about this analysis. 

First, the author seems to make the implicit assumption that all statisticians believe in the IID assumption. The IID assumption is an example of a statistical model that we may use to fit baseball data. Any model we apply is actually wrong—that is, the real process behaves in a much more sophisticated manner than the model suggests. For example, take the standard IID assumption that individual at-bats are coin-tossing outcomes with a constant probability of hitting success p. Do I believe this is true? Of course not. I believe that the hitting talent of a player goes through many changes during the season and it depends on many other variables such as the quality of the pitcher, the game situation, whether the game is at home, etc. So if the IID model is wrong, why do we use it? Well, the IID model has been shown to be useful in understanding the variation of baseball data. One thing that I have found remarkable in my baseball research is that good simple models (like the IID model) are really helpful in predicting future baseball outcomes. 

Second, the author gives the impression that this statistical analysis gives evidence for the hot hand effect in baseball. Suppose you reject the IID assumption—what does this mean? It could mean that there is a dependence structure in the batting sequence. That is, one’s performance in one at-bat is helpful in predicting the performance in successive atbats. But there is a second possible explanation. Maybe the outcomes are independent, but the chance of getting a hit changes across the season. Either explanation, a dependence pattern or a change in hitting probability, would explain the presence of long streaks. Also these two characteristics are confounded and it is difficult statistically to isolate their effects. So it is wrong to say that long streaks imply a dependence pattern in the hitting sequences. People love to believe in the hot hand and I’m concerned that this paper adds fuel to their hot-hand belief. 

Last, what is the practical significance of the results? To find these streaky effects, the author had to consider all hitting sequences in 50 seasons of baseball data. This is a ton of data—the author was likely considering streaks present in over 30,000 player seasons! But we live in the context of a single season and these results really don’t say that the IID assumption is inappropriate for understanding the lengths of hitting streaks for a single season. I suspect that in the context of a single season, these streaky effects are relatively small and can safely be ignored. The author is concerned about the difficulty of devising a more accurate modeling method since he has shown that the IID assumption is incorrect. But that’s okay, since statisticians don’t need “exact” models. If the streak effect is “real” but small in size, then I’ll continue to use the IID model since I believe it is an attractive approximate model that works. 

Is There Evidence of Long Streaks for the Past Five Seasons?

After reading this paper, it seemed natural to explore the presence of long streaks in the context of a single season. McCotter demonstrated that there was a streakiness effect, but didn’t measure the size of this effect. If the streakiness effect was substantial, then I would think it should manifest itself in a single season. 

So I replicated the author’s analysis for each of the five recent baseball seasons from 2004 through 2008. I’ll carefully outline what I did for the 2004 season which may help explain the author’s method in the BRJ article.

-Using play-by-play files from Retrosheet, I collected the game-to-game hitting data (number of hits and number of at-bats) for all 959 players who had at least one official at-bat in the 2004 season.

-For each player’s game log, I collected the lengths of all hitting streaks. For example, for the 2004 John McDonald, I record if he got a hit (Y) or not (N) for each of the 40 games he had an official at-bat in the season. (See table 1.)