Literature Search

So you’re at the point where you have a research idea in mind. Your next step, then, is to find any previous work that’s already been done on your topic.

In academia, there’s a conventional wisdom on how to do a literature search, and a lot of it involves indexes to scholarly journals that cover your topic. In sabermetrics, however, it’s not quite so simple — much of the best research is published online, on any one of hundreds of websites, without a formal peer-review process to separate the good from the flawed.

So, as much as we might wish there were a step-by-step process for finding existing work, the reality is that it becomes a bit of a seat-of-the-pants thing. Some suggestions, though, for how to proceed:

1. Scan the research repositories

While most sabermetric work of recent vintage is web-published, there are still several more formal repositories of studies. The advantage of those is that, if they’re all at one specific website, you can search them online by using any normal search engine (such as Google), but using the “advanced search” feature to ask for results only from that one site.

Some specific places to look:

• Every back issue of SABR’s “By The Numbers” is available. There is a repository at the SABR website and at my own website, www.philbirnbaum.com.
• In the 1980s, Bill James edited the “Baseball Analyst,” a sabermetrics newsletter that went out to what I think were only a few dozen subscribers. In 2012, SABR published those online for the first time at sabr.org/research/baseball-analyst-archives.
• Tom Tango, one of the leading active sabermetric researchers today, has some of his own studies at his website, tangotiger.net.
• Tango and his co-authors of “The Book” have set up a wiki, an open-source encyclopedia of sabermetric subjects. There has been some talk of abandoning the project, but, at time of writing, it’s still active at tangotiger.net/wiki/index.php?title=Main_Page.
• Charlie Pavitt, a SABR member and regular contributor to “By the Numbers,” has compiled a bibliography of published sabermetric papers. It’s dedicated to only the more formal publication outlets, so it’s missing a large part of the recent explosion in web research. Still, it’s a worthy source. A description can be found here and the bibliography itself can be found here.

2. Search the biggest websites dedicated to sabermetric research.

My advice would be to start by searching “The Book” blog. There, Tom Tango reviews, or at least mentions, a large proportion of the most significant studies. Also, the site has, in my opinion, the densest population of knowledgeable commenters; almost always, you learn more from the comment discussion than from the studies themselves. Comments do show up in the searches, I believe.

From there, consider these other sites:

• The Hardball Times
• Baseball Prospectus (subscription required for some content)
• Baseball Analysts
• Beyond the Box Score
• FanGraphs
• TangoTiger

In 2010, Beyond the Box Score held a poll to vote for the best sabermetric websites and studies. All the nominee websites are worth a look and a search, and can be found at http://www.beyondtheboxscore.com/2010/1/21/1263306/your-btb-sabermetric-award-voting.

3. Ask.

Perhaps the best way to find research on a certain topic is to ask around. There are various places to ask, but, before doing so, please spend some time looking for yourself. That’s just a courtesy to those to whom you are requesting assistance. I have had people e-mail me about finding research on topic X, when they could have found what they’re looking for by doing the simplest search for “X” on Google.

People are generally very willing to help when you show what you’ve tried, and you let them know what you’ve found so far.

Places to ask:

• One good bet is to write to authors of studies on topics that are close to yours. If you’re thinking of doing a study on how accurate scouts are when they evaluate pitchers, and you find a study on how accurate scouts are when they evaluate batters … well, the author is probably as interested in the subject as you are, and is likely to be able to help. Even if the answer is, “sorry, I don’t know of anything,” that’s a sign that your topic may indeed be a fresh one.
• Most websites allow comments on the studies they publish. If there’s a topic that’s similar in some ways, post a comment asking about your topic.
• Ask on e-mail forums. SABR has SABR-L, which is probably a bit too general for many detailed sabermetric inquiries, but still worth a shot. A better place is the Yahoo group “statisticalanalysis,” which is free to join for SABR members with an interest in sabermetrics.
• Finally, you can try specific people. I don’t mind an occasional inquiry, and I’m sure many others are happy to answer too. If you’re stuck, you can always try writing to someone who you know is an active researcher. Many of the sabermetric websites have links to contact authors. Sabermetrician John Doe may not have published anything that touches on your specific topic, but if he publishes a column every week and a research paper once a month, you wouldn’t be out of line occasionally sending a courteous request for assistance.