Cameras and Computers, or Umpires?

According to baseball rules: ”A STRIKE is a legal pitch when so called by the umpire, which is not struck at if any part of the ball pass though any part of the strike zone.

“The STRIKE ZONE is that area over home plate the upper limit of which is a horizontal line at the midpoint between the top of the shoulders and the top of the uniform pants and the lover level is a line at the hollow beneath the kneecap. The strike zone should be determined from the batter’s stance as the batter is prepared to swing at a pitched ball.” 

Simply pictured, any pitched ball that would touch an imaginary vertical extension of the five-sided home plate extending above a horizontal plane at the knees and terminating below a horizontal plane at the letters, as defined by the rule book, is a strike.

For more than a century an umpire (human by statute if not in the view of some fans) has called balls and strikes. With modern technology, fast video cameras and fast computers analyzing the video images, that determination can be made instrumentally. Such systems have been used in televised game broadcasts for some time. In 2002, Major League Baseball arranged for the operation of the QuesTec™ system in the Anaheim, Arizona, Boston, Cleveland, Houston, New York (Shea), Milwaukee, and Tampa Bay ballparks to evaluate umpires in a program labeled the Umpire Information System (UIS).

All such systems work in approximately the same manner-tracking the flight of baseballs as they near the plate. This reporter has been given access to detailed technical data on one such system as well as rather less information on QuesTec™, which is closely held. Also, my extensive experience in the computer analysis of photographs of tracks of elementary particles moving through liquid hydrogen has been surprisingly useful in giving me insights into baseball-track reconstruction problems.

The sketch of Figure 1 suggests the positions of the cameras in a typical UIS system. The primary video cameras, C-L and C-R, are mounted high in the grand stand overlooking first and third base. Auxiliary cameras (labeled c in the diagram) are placed low, often near the dugouts, and are used to set the high and low limits of the strike zone at the letters and knees of each batter. Constrained by the ball park architecture, the cameras are placed somewhat differently in each park.

Using the limited information I have about QuesTec™, and more extensive information about a similar system, I can reconstruct the UIS measuring process in a manner that cannot be far from the mark. Each of the two primary cameras takes a “picture” of the ball every I/30th of a second as it nears the plate. The resultant images reflect the position of the ball at intervals of the order of four feet over about the last 30 feet of the ball’s flight until it nears the plate. Blocked by the batter, the ball is not actually tracked over the plate but the ball position at the plate is calculated extrapolating from the measurements of its approach trajectory.

The image planes of the cameras used for baseball tracking are typically made up of three sets (for brightness, red, and blue) of CCDs (charge-coupled devices) with 758 (vertical) x 494 (horizontal) pixels. The image of the 2.9″ diameter ball is then roughly 8 pixels vertically and perhaps IO horizontally as the ball image is elongated about an inch by the motion of the ball during the 1/IO00th of a second “exposure.” There is no physical shutter; the CCDs are read electronically. The information from the image planes of the two cameras is then processed by the computer to construct the position of the ball as it moves toward home plate. The position of the center of the ball at each picture time can be reconstructed from the images captured by the two cameras to an accuracy of about “horizontally and 3/16” vertically. In this reconstruction, the computer corrects for some classes of distortions, such as barrel or pin-cushion lens distortions.

QuesTec™ has made some measurements of the accuracy of their system that-according to their reports-appear to be quite well done, if somewhat limited. After a preliminary set of experimental measurements at Tampa, they tested their system at Fenway, where they measured the differences between the calculated and measured impact points of 44 pitches that hit a backboard at home plate. Measuring only the random variations, they found a “root-mean square” (average) left-right error of 0.40 inches and an up-down error of 0.47 inches.

This assessment of error did not include some systematic errors such as errors in the calibration of the system during a game, which can be significant. The extensive calibration set up for the Fenway test was quite special and not used later in real-game situations. Nor was the data sample sufficiently large to evaluate the likelihood of rare large errors. If the “random” errors were made up of very many independent small errors, the probability of large errors is well-defined; only about one pitch in 80 would have a left-right error from random fluctuations greater than an inch, and one in 25 an up-down error greater than an inch. However, in most real measurements, the probability of large deviations — the “tails” of the distribution — are greater than expected from a single normal distribution. Nevertheless, I would expect that the probability of a random error greater than two inches would be negligible.

However, most complex systems make occasional “mistakes” as well as errors, and mistakes are not easily tractable. My own experience of a half-century of experimental physics tells me that while it is no inconsiderable effort to construct a system such as QuesTec™ which operates correctly 98% of the time, it is often the very devil to get to 100%.

During the year 2002, UIS was used to check the ball-strike calls of plate umpires.

The system requires an operator who uses two computers and several monitors. Before the game, he calibrates the system-that is, locates home plate-through a defined procedure using one of the monitors. He may repeat the calibration during the game. As each pitch is made, the operator records the disposition of the pitch on the scoring computer, pressing buttons that signify “swung at,” or “called by the umpire.” Warm-up throws, etc., are automatically recorded by the system but elicit no input label. There is also a button that signifies “bad track,” presumably pressed when the system fails transparently. The path of each pitch, as determined by the tracking cameras, is recorded by the system and displayed on a monitor. A second monitor shows the game from the center-field camera.

After the game the operator views the pictures taken, using the ground-level cameras, of the batters at the plate in one of the monitors and sets the high and low strike levels at the knees and letters of each player as the player stands waiting for a pitch.

With the upper and lower limits of the strike zone defined for each batter, the operator sets up the computing program that calls balls and strikes for each pitch and reviews each pitch. At this time he identifies “bad tracks; where the computed track is clearly not in accord with other information. Then a CD is produced for the plate umpire that shows the computed ball tracks and the system and umpire ball-strike calls for each pitch with a letter C (correct) if the umpire and UIS give the same call, A (acceptable) if the calls disagree but a change of two inches in the computed trajectory would bring the calls in agreement, and N (not acceptable) if the calls disagree by greater than two inches. The operator manually excludes the information on the “bad track” pitches that he identifies, which are usually, if not always, labeled N by the system.

This system was used to evaluate umpires in about 600 games in 2002. A total of 83,891 pitches were recorded where both the umpire and UIS called balls and strikes. Man and machine agreed ( C) on 71,164. They disagreed (A) on 4,970 where the trajectory difference was less than two inches, a difference that was considered within the uncertainties in the system, and hence the umpires’ calls were acceptable. However, they strongly disagreed (N) on the call-by more than two inches in the trajectory-in 7,757 pitches. Thus man and machine differed strongly on about 9% of the called pitches, for an average of about 14 pitches per game.

There were differences in the scores of the 79 different umpires. I made a statistical analysis of those scores and found that the differences between umpires were almost wholly due to chance fluctuation in the scoring process and not from whatever differences there might be in the umpires’ ball-strike judgments. Although the umpires believe that they had been told in the Major League Baseball Umpire Manual that the 2002 results from the UIS system were to be used only for “training to improve performance” and “no umpire will be judged . . . for the 2002 season;’ they believe that their scores according to UIS were considered in the appointment of umpires for the post-season series.

The errant calls are classified in the table where U-s, Q-b means that the umpire called the pitch a strike, and the computer called it a ball. Conversely, U-b, Q-s stand for pitches that the umpire called balls and the computer strikes.

It is evident that the differences were not random, and hence that most of the N (not acceptable) umpire calls were not simply due to erratic judgments. As the most extreme discrepancy, the computer called 2,007 low pitches strikes that the umpires called balls, while the umpires called only 18 low pitches strikes that the computer judged were balls. Clearly the computer strike zone extended far below the umpires’ zone.

With the aid of a plausible model, I have estimated the difference between the strike zones as shown in the cartoon of Figure 2. Independent of which is “right” or even if there is any precise meaning to “right.” UIS and the umpires call different strike zones. The computer zone is much narrower than the umpire zone but longer in the vertical direction. That difference between zones accounts for about half of the 9% of pitches where man and machine differ. If the umpires would adjust by calling balls much tighter on the outside corner, a little tighter on the inside corner, allowing a slightly higher strike, and calling almost any low ball that isn’t in the dirt a strike, their UIS error score would decrease by about 4.5%.

Moreover, a substantial fraction of those errors would probably be from the QuesTec™ system operator plus machine. While the rate seems to vary widely game-to-game and park-to-park, the operators seem to have thrown out an average of five pitches per game (or about 3% of the pitches) as ”bad tracks;’ QuesTec™ system mistakes. How many of the 4.5% residual

errors were then actually less dramatic QuesTec™ mistakes rather than umpire errors? Perhaps almost all! Indeed, the umpires contend that the UIS system miscalls occasional pitches by more than a foot and sometimes much more.

Though the umpires could adjust to the UIS strike zone, such an umpire adjustment would change the game of baseball. The pitch on the outside corner now called a strike, that many pitchers (e.g., Greg Maddux) live by, would be gone. Conversely, those pitchers that like to throw high and hard together with a splitter low and slow (e.g., Hideo Noma) might do very well.

Baseball, in the form we know, is a game with an important tradition that goes back more than a century. My grandfather Ted Wiegman, who played sandlot baseball at third base in Fort Wayne, Indiana, of the 1890s, if alive would have no trouble identifying the modern game today with the game he loved as a youth. If somehow umpires (all umpires!) have in recent years drifted away from the traditional game and dramatically changed the strike zone, surely it would be desirable to bring it back to the letter of the rule book, and the UIS could assist in doing that. Indeed, with only a little more development we can envision balls and strikes called only by machine with the umpires relegated to judging base-running plays and such.

However, if the present umpires’ strike zone is really the traditional zone, called that way long ago by umpires like Jocko Conlon, Bill Klem, and Cal Hubbard, we should be wary about changing the zone and the game. I have made some computer simulations that suggest to me that the umpires’ zone is largely the natural human interpretation of the rule book zone and, most likely, that zone has always been as it is called now. In the future, highly developed machines could surely call balls and strikes more accurately than human umpires (and pitching machines can throw harder than Randy Johnson!), but for the time being, let’s keep humans judging and playing this human game.

ROBERT K. ADAIR is Sterling Professor Emeritus of Physics at Yale University and a member of the National Academy of Sciences. His research has largely been connected with the properties of the elementary particles and forces of the universe. At the request of his friend Bart Giamatti, he served as official Physicist to the National League 1987-1989.