An Examination of MLB Play Call Challenges

The replay review system has changed Major League Baseball. The goal of the instant replay system was to reduce the impact of umpire error, while minimizing the time needed to review plays. In this paper we will examine the effects that replay review has had on the game and its strategies.

Major League Baseball (MLB) was the last of the four major US sports to implement a replay review system. Opposition to replay review included arguments that “bad calls” are traditionally part of America’s national pastime, and concerns over game delays and the limitations of instant replay technology.1 Despite strong resistance, MLB implemented a limited replay review system in 2008.2 The initial system mandated that umpires on the field select play calls to be reviewed rather than team managers. The limited replay review system was in effect through 2013.

In January 2014, MLB announced an expansion of the replay review system.3 The expanded version extended play challenges to team managers and increased the types of plays that may be reviewed.4 The new rules, which are still in effect as of this writing, allow each team manager two challenges to start each postseason game, divisional or wild card tie-breaker game, and one challenge to start every other game including the All-Star Game. If a challenge is upheld, the manager retains the ability to challenge another play call, but never more than two in a regular season game.5 The umpire crew chief also has the discretion to review a potential home run call at any time during a game. Beginning in the eighth inning, a crew chief may initiate a play call review of any other reviewable call if the team manager has no challenges remaining. To accommodate the expected increase in play call challenges, seven minor league umpires were promoted to the major leagues and the Replay Operations Center was established.6

The new rules brought about changes in the staffing at the team level as well. MLB authorized teams to hire video review coordinators. A review coordinator would quickly assess footage from multiple camera angles and counsel team managers whether to challenge a play.7 The review coordinator is largely invisible to spectators, but two additional changes are noticeable to those watching a game. First, the “dirt-kicking, cap-flipping, vein-popping argument” that has been part of baseball’s tradition of challenging play calls has been replaced by a modern watch party, as predicted before the 2014 season by Paul White in USA Today.8 Second, base running is arguably more physical as “the swipe, pop and phantom tags are disappearing because they are less efficient at guaranteeing contact with the runner.”9

For sabermetric enthusiasts, of course, a new replay review system means new measures of performance for replay review analysis. In the spring 2015 issue of the Baseball Research Journal, Gil Imber presented three new measures of performance including the Review Affirmation Percentage (RAP), Team Success Percentage (TSP), and Manager’s Challenge Success Percentage (MCSP). RAP is the percent of play calls affirmed and thus not overturned. RAP includes analysis of play calls reviews challenged by both umpires and team managers. TSP is the percent of play calls overturned regardless if the play call review was challenged by umpires or a team manager. MCSP is the percent of play calls overturned when the play call was challenged by the team manager.10

PURPOSE OF THE STUDY

The purpose of this study was to examine MLB play call challenge data (RAP, TSP, and MCSP) in relation to a variety of variables. Data from play call challenges (N=1352) during the 2015 MLB season were analyzed across the 30 MLB teams, the types of play calls reviewed (home run, ground rule/automatic double, fan interference, stadium boundary calls, force play, tag play, fair/foul on balls hit in the outfield, trap play in outfield only, batter hit by pitch, timing play. touching a base, passing runners, and record keeping), umpire positions, and innings the play was challenged. Specific areas examined included the relationships between the MCSP and teams, RAP and types of play calls, RAP and umpire positions, and RAP and the inning the play was challenged.

Method

Data from play call challenges during the 2015 MLB regular season (N=1352) were obtained from baseballsavant.com in January of 2016. Specific play call challenge variables included MLB team challenger, the types of plays reviewed (catch or drop, fair or foul on balls hit in the outfield, fan interference, ground rule/automatic double, hit by pitch, home run, home-plate collision, play at first, record keeping, rules check, stadium boundary, tag play, tag-up play, timing play, touching a base, and trap play), position of the umpire making the call (first base, second base, third base, and home plate), and the inning the play was challenged (1, 2, 3, 4, 5, 6, 7, 8, and 9+). Once the data were coded, entered and cleaned (duplicated challenges were excluded), they were analyzed using Statistical Analysis Software (SAS). A variety of analyses were used to generate descriptive data and examine trends related to MCSP and MLB teams, RAP and types of play challenges.

Results

MCSP and Teams

Of the 1352 play call challenges, 50.74% were overturned (TSP = .5074; RAP = .4926). The umpires challenged 172 play calls, more than any individual team. However, the umpires had the highest RAP of .7126. It is important to note umpire challenges apply to all games, while the manager challenges only apply to that team’s games. Therefore, although the umpires are over four times higher than the team average, they are only about 1/7 of the total number of challenges. The Rays challenged the most play calls of any team (n = 53) and the Tigers challenged the least (n = 27). The mean of number of challenges per team was 44.61. The Yankees had the largest percent of play calls overturned (75%; MCSP = .75) and the Rays had the fewest (32.08%; MCSP = .3208). The mean MCSP was .5440 (see Table 1).

 

(Click image to enlarge.)

 

A Chi-square was conducted to examine the differences in MCSPs among MLB teams.  MCSPs did statistically differ among MLB teams, X² (30, N = 1352) = 86.25, p = <.0001. As shown in Table 1, The Yankees, Mariners, Diamondbacks, Royals, and Twins were more likely to have a play call overturned than the Rangers or Rays.

A correlation analysis examined the relationship between the number of play calls challenged and MCSP values. The number of play calls challenged by a team were moderately negatively correlated with MCSP (r = -.495). Therefore, the analyses indicated the more plays a team challenged the lower the MCSP value.

Seventy-three percent of play calls reviewed were either tag plays (40.75%) or plays at first (32.32%). During the 2015 MLB regular season, there was only one double reviewed and two timing plays reviewed. The type of play call most frequently overturned was fan interference (66.66%; RAP = .3333). Rules check, timing plays, and automatic double types of plays were not overturned during the 2015 regular season.

A Chi-square was used to examine the statistically significant differences in RAP values among types of play calls reviewed. RAPs did statistically differ by type of play reviewed, X² (16, N = 1352) = 94.28, p = <.0001. As shown in Table 2, play calls on fan interference, plays at first, force plays, and hit by pitch were more likely to be overturned than rules checks, timing rule, automatic double, home-plate collision, or tag up play calls.

 

(Click image to enlarge.)

 

RAP and Umpire Position

The position of the umpire that was reviewed the most frequently was first base (39.94%). A Chi-square examined the differences between RAP values among umpire positions.  RAP values did statistically differ among umpire positions, X² (3, N = 1352) = 50.31, p = <.0001. Umpires positioned at first base (RAP = .3981) were more likely to be overturned than umpires positioned at third base (RAP = .6529) or home plate (RAP = .6053; see Table 3).

 

(Click image to enlarge.)

 

RAP and Inning of Play Call

More plays were reviewed during the seventh inning (n=189) in the 2015 regular season than any other inning. The fewest number of plays — 119 — were reviewed during the fourth inning. Plays reviewed in the first inning had the highest percent of calls overturned (66.01%; RAP = .3399) and plays reviewed in the eighth inning had the lowest percent of calls overturned (53.33%; RAP = .6467; see Table 4). A Chi-square indicated RAP values did statistically differ among innings the play call was reviewed, X² (8, N = 1352) = 44.96, p = <.0001. Play calls challenged in the first, second, or third inning were more likely to be overturned than plays challenged in the eighth or ninth. (See Table 4).

 

(Click image to enlarge.)

 

A logistical regression analysis examined the relationship between the inning the play was challenged and RAP values. The regression model was adjusted for MLB team and umpire position. What inning the play call was challenged significantly predicted RAP scores, t(1317) = -6.19, p < .0001. The inning of a challenge also explained a significant proportion of variance in RAP scores, R² = 0.1358, F(1,1317) =38.27, p < .0001.

Discussion

Our 2015 MLB replay review data analyses bear similarities to analyses of the 2014 replay review data.11 Imber stated that based on the 2014 MLB replay review data, “the more reviews experienced by a given team, the greater the chance that some of these reviews were unsuccessful and/or frivolous.” Similarly, this study indicated a negative correlation (r = .495) such that the more play calls a team challenged, the lower the team’s MCSP value—and thus, lower the likelihood the review would be overturned. In both the 2014 and 2015 regular seasons, the umpires had the highest RAP at .769 and .7126, respectively. These trends may in part be due to the choice of teams to request a review from the umpire rather than use a manager’s challenge. Teams more successful at getting umpires to initiate a play call review would have a lower frequency of manager play call challenges. Moreover, it may be assumed that teams would be more likely to request the umpire initiate a review rather than use a manager’s challenge when the play call in question is indeed very questionable.

Another trend from the 2014 MLB regular season that carried over to 2015 was that the RAP increased as the game progressed.12 There are several possible reasons for this. Umpires may have improved as each game advanced. Managers’ judgment in challenging play calls may have gotten poorer as each game continued. However, the most likely reason is that to use a manager challenge early is the game is a greater risk than using one later in the game. Play challenges can also be used as part of a strategy to provide more time for a pitcher during high leverage situations in the game.

The MLB replay review system has changed some aspects of America’s favorite pastime.  It has added a new element to team strategy, new team staff positions, and new data to be analyzed. Data from the 2015 MLB regular season indicated RAP is impacted by team, type of play call, umpire position, and most statistically significantly, inning the play was challenged. Teams should continue to invest in understanding and maximizing the opportunities presented by the replay review system.

ANNE C. MARX SCHEUERELL is an Associate Professor of Sport Management in the Francis J. Noonan School of Business at Loras College in Dubuque, Iowa. Her research is in sport analytics and sport as a platform for socio-cultural change. She earned her master’s degree from Arizona State University and her doctorate degree from the University of Arkansas.

DAVID B. MARX is a Professor Emeritus of Statistics at the University of Nebraska in Lincoln, Nebraska. He works in the area of spatial statistics as well as in sports statistics. His Ph.D. is from the University of Kentucky and he was previously employed at the University of Arizona and University of Arkansas.

 

Notes

1 Zachary D. Rymer, “Breaking Down the Pros and Cons of Instant Replay in MLB.” Bleacher Report. October 03, 2017. Accessed September 19, 2018. http://bleacherreport.com/articles/1156392-breaking-down-the-pros-and-cons-of-instant-replay-in-mlb.

2 Associated Press. “MLB Approves Replay in Series That Start Thursday.” ESPN. August 27, 2008. Accessed September 17, 2018. http://www.espn.com/mlb/news/story?id=3554357

3 Paul Hagen, “Expanded replay approved, to begin this season.” MLB.com. January 16, 2014. Accessed February, 2015 from http://m.mlb.com/news/article/66737912/mlb-approves-expanded-instant-replay-beginning-with-2014-season

4 MLB Replay Review Regulations. (n.d.) Accessed February 25, 2019. http://m.mlb.com/official_rules/replay_review.

5 Hagen, MLB.com, 2014.

6 Gil Imber, “Confirmed: MLB Hires 7 New Umpires to Full-Time Staff,” Close Call Sports & Umpire Ejection Fantasy League. Accessed September 19, 2018. http://www.closecallsports.com/2014/01/mlb-hires-umpires-baker-blaser-rackley.html.

7 B. Walker, “Replay Wizards Becoming Key Positions on MLB Teams.” Spokesman.com. March 21, 2014. Accessed September 19, 2018. http://www.spokesman.com/stories/2014/mar/22/replay-wizards-becoming-key-positions-on-mlb-teams/.

8 Paul White, “How Instant Replay Will Truly Change Baseball,” USA Today. February 28, 2014. Accessed September 19, 2018. http://www.usatoday.com/story/sports/mlb/2014/02/27/how-instant-replay-will-change-baseball/5879639.

9 David Manel, “How Has Instant Replay Changed Baserunning?” Bucs Dugout. May 20, 2015. Accessed September 19, 2018. http://www.bucsdugout.com/2015/5/20/8626635/tagging-sliding-replay.

10 Gil Imber, “Reviewing Instant Replay: Observations and Implications from Replay’s Inaugural Season.” Baseball Research Journal 44, no. 1 (2015). Accessed September 19, 2018. https://sabr.org/research/reviewing-instant-replay-observations-and-implications-replay-s-inaugural-season.

11 Imber, “Reviewing Instant Replay,” BRJ, 2015.

12 Imber, “Reviewing Instant Replay,” BRJ, 2015.