Big Data and Baseball–‘The Shift’

I grew up in the Golden Age of baseball; or should I say the traditional era.  No designated hitters, no challenges and video reviews, few relief pitchers, and no infield shifts.  Baseball was an elegant, timeless game with nine players deployed in fixed positions, adjusting to accommodate left-handed and right-handed batters, but basically deployed as they had been since Abner Doubleday.

Baseball was a big part of my childhood.  As kids we played pickup on the New Brighton Green, practiced our pitching with a tennis ball on the garage door, played Wiffle Ball, and above all listened to the games on Saturday radio.

While my sister ran in the sprinkler and watching my father spray the aphids on his rose bushes, I listened to Mel Allen, Red Barber, and the Yankees.   We traded baseball cards, argued whether Mickey Mantle or Ted Williams was the greatest ballplayer of all time, and batted rocks out by the quarry.

Baseball has always been a game of statistics, and as long as the game didn’t change, one could compare the performance of any player against that of any other who played before him.  There were anomalies, of course – the dead ball era, for example, when in 1919 the great Babe Ruth could only whack 19 home runs instead of his record 60 eight years later and after the ball was lightened.  There was the era of the juiced ball many decades later.  Although there was no conclusive proof that Major League Baseball tinkered with the ball to produce more offense, home runs were hit at a record pace. Much more recently there was the steroid era when Babe Ruth’s and Roger Maris’ 60 and 61* home run records were surpassed by a long shot – and not by just one hitter.

However, knowledgeable baseball fans take all this into consideration and discount or revalue the achievements of athletes who played in eras that skewed the game.  For them, statistics are still a robust link to even the distant past of the game.

The first thing I read in the paper in the morning are the box scores.  I keep track of my favorite players, particularly the ones who perform at levels way beyond their talented peers. Miguel Cabrera of the Detroit Tigers is an uncanny hitter.  He simply doesn’t miss and has baseball’s special gifts – vision, quickness, power, coordination, patience, and concentration.  Athletes like Cabrera come along in other sports as well, usually once a generation, and they are compelling to watch when they are in their prime. 

Tiger Woods is a good example.  He was elegance and majesty on the links.  No other golfer before him was as uniquely gifted and talented.  Both Jack Nicklaus and Arnie Palmer, great in their own times, wondered from what distant planet he came from.  Wayne Gretzky could outskate, outmaneuver, and outthink every other player on the ice.  Even when opponents thought they had taken the measure of the man, he outsmarted them.  Pete Sampras won an almost unimaginable number of major tennis titles, a record that would last for decades; but then along came Roger Federer, thought by many to be the best, most complete, and most artistically talented of all.

Old-fashioned as I am, I stick to the box scores in the daily paper – runs, hits, errors, ERA, and W-L. Real baseball buffs go to Baseball Digest which collates and presents data in many new measurement categories - OBP, OPS, PA/SO, RISP and many others.  More and more is known about each player, and given the ease with which performance can be tracked and recorded electronically, probability numbers are more and more accurate.  It is easy to determine where a given player is likely to hit a ball given the pitcher, the stadium, the weather, and the time of day. A manager with a few clicks of his mouse can adjust his defense for each batter.

No longer are infielders and outfielders deployed in traditional ways, but arrayed according to the Theory of the Shift.  If a batter has hit the ball on the ground to the right side of the infield 80 percent of the time, then the entire defensive alignment changes.  Everyone moves right towards first base, leaving a gaping hole between third and second base.  Of course The Shift works – numbers don’t lie – and batting averages which have been going down since teams have employed The Shift confirm it.  The batting average for balls hit on the ground has been even lower since infielders have been stacked on either the left or right side of the diamond.

When Big Papi (David Ortiz of the Boston Red Sox) comes to bat, fans know that he may strike out, hit long fly balls to the outfield, or launch a home run; but they are pretty sure – given his statistical history – that he will hit something hard to the right side of the infield.  In years past, these shots usually found a hole between the first and second baseman, and Ortiz’ average soared.  Now Big Papi whacks the ball as hard as he ever did, but because of the asymmetrical alignment of the defense, the rocket is hit right to someone.  In the Big Papi Shift, not only do players all move towards the right field line, the second baseman plays short outfield.  If Ortiz does what the probability suggests, he is a goner.

“Not fair!”, yell the Fenway Faithful, Billy Ball and Nate Silver robbing Big Papi of winning hits.  “The game wasn’t meant to be played this way.” 

It surely seems that electronic statistical analysis has favored the defense; and for the time being it will continue to do so.  Pull hitters have been hitting that way since Little League, and are unlikely to change. Their mechanics have been tooled to hit the ball hard in one direction, and they are paid to do so.  Pull hitters are usually home run hitters, because they generate so much torque in their left- or right-field swings, and RBI’s win games.  Getting Big Papi to hit to all fields would be an impossible and unfair request.

However, scouts will now begin to put a premium on all-field hitters – young Ichiros, George Bretts, and Tony Gwynns. Within a few years The Shift will disappear as this new crop of talented hitters smack the ball wherever the pitch dictates.  Baseball will return more than ever to its roots and to its elegant simplicity/complexity.  Home runs will go down – and MLB owners never like that – so eventually pull hitters and a modified Shift will return; but for a while, anyway, baseball purists will be able to see great, all-around hitting and well-timed defense.

The Shift had to come. After years of analyzing numbers but never doing much with them, managers have now made a significant change to the way the game is played.  They finally can put statistics into practice.

Big Data has revolutionized marketing, scientific inquiry, and punditry.  Nate Silver is now far more important than George Will, David Brooks, and Charles Krauthammer combined.  He gets elections right every time, and they never do.  Google, endowed with a stable of brilliant engineers in the Googleplex, still goes to crowdsourcing for improved search engine algorithms.  Noam Chomsky, renowned for his work on understanding intelligence and brain function, has been bested and one-upped by Big Data companies.  In discussions on Artificial Intelligence Google researchers have rejected Chomsky’s claim that in order to build intelligent machines an understanding of the human brain is necessary.  Nonsense, they say.  One only needs to look at the way language is used and intelligent programs can be devised accordingly.  Focus groups are out, and crowdsourced surveys are in.  Everything is being rethought.

If anyone doubts the revolutionary nature of Big Data, they only have to go to Camden Yards and watch the Orioles play.  They are the team that in 2013 employed the shift more than any other – over three hundred times.  Baseball analysts suggest that that number could easily exceed 1000 in the coming years.

I am not at all upset by The Shift because it was inevitable.  I am also not upset because in baseball’s competitive environment offense and defense will continue to fight for turf and territory, and equilibrium will always be re-established.