The Flaw of Sabermetrics: How WAR Can Ruin Payrolls
I bought in. I did research. I studied numbers and related them to the players, while developing a deeper understanding of the game. All of the random numbers that took the place of a batting average, home runs, RBI, ERA, and wins in the evaluation in baseball stole my heart. Sabermetrics gave creative, number-crunching baseball fans a means to display defensive values, base running values, and even what a pitcher could do without that guy with stone hands playing shortstop. They even provided a way to determine how Roberto Clemente matches up to Vladimir Guerrero thanks to ballpark adjustments, allowing statistics to see what a player from the past could do today.
All of that is great. Fine. Dandy, as my late grandpa would say. However, for every xFIP, FIP, wRC+, and OPS+, there is WAR – Wins Above Replacement. What is WAR exactly? Well, it’s funny that I ask myself such a question and then provide the answer.
FanGraphs has a nice explanation:
Offensive players – Take wRAA, UBR & wSB, and UZR (which express offensive, base running, and defensive value in runs above average) and add them together. Add in a positional adjustment, since some positions are tougher to play than others, and then convert the numbers so that they’re not based on league average, but on replacement level (which is the value a team would lose if they had to replace that player with a “replacement” player – a minor leaguer or someone from the waiver wire). Convert the run value to wins (10 runs = 1 win) and voila, finished!
Pitchers – Where offensive WAR used wRAA and UZR, pitching WAR uses FIP. Based on how many innings a pitcher threw, FIP is turned into runs form, converted to represent value above replacement level, and is then converted from runs to wins.
Then, Baseball Reference has a deeper explanation – first for position players:
WAR for position players has six components:
- Batting Runs
- Baserunning Runs
- Runs added or lost due to Grounding into Double Plays in DP situations
- Fielding Runs
- Positional Adjustment Runs
- Replacement level Runs (based on playing time)
The first five measurements are all compared against league average, so a value of zero will equate to a league average player. Less than zero means worse than average and greater than zero better than average. These five correspond to the first half of our equation above (Player_runs - AvgPlayer_runs). The sixth factor is the second half of the equation (AvgPlayer_runs - ReplPlayer_runs).
They also have an explanation for pitchers at Baseball Reference, starting with a basic idea:
At its most basic level, our pitching WAR calculation requires only overall Runs Allowed (both earned and unearned) and Innings Pitched. Since we are trying to measure the value of the pitcher’s performance to his team, we start with this runs allowed and then from there adjust that number to put the runs into a more accurate context.
But it doesn’t stop there! Baseball Prospectus has WARP – Wins Above Replacement Player, which they consider:
Wins Above Replacement Player is Prospectus’ attempt at capturing a player’ total value. This means considering playing time, position, batting, baserunning, and defense for batters, and role, innings pitched, and quality of performance for pitchers.
Perhaps no sabermetric theory is more abstract than that of the replacement-level player. Essentially, replacement-level players are of a caliber so low that they are always available in the minor leagues because the players are well below major-league average. Prospectus’ definition of replacement level contends that a team full of such players would win a little over 50 games. This is a notable increase in replacement level from previous editions of Wins Above Replacement Player.
Here is an example of the Wins Above Replacement Player spectrum based on the 2011 season:
With so many different versions of player value calculations, how do you determine which one is most appropriate in truly determining player values?
Even if you have a preference of which WAR or WARP system to use for your ranking or player value thought processing, why are you using it, but most importantly, why would a Major League Baseball team consider using WAR when discussing player contracts – now or in the future?
Teams are getting surplus value out of the pre-arbitration and arbitration years of their team-controlled talent, prior to the players cashing in with the exorbitant figures that seem to be getting thrown around on the free agent or long-term extension markets. Clayton Kershaw‘s seven-year, $210 million deal was likely the stepping stone to several future $30 million or more average annual value (AAV) deals in the near future. Based on Kershaw’s 18.5 WAR (FanGraphs) over the last three seasons, he is the epitome of excellence on the mound – a modern day Sandy Koufax in the familiar Dodgers uniform…without the ice baths and elbow pains. You could assume that no pitcher currently in MLB is worth more than Kershaw due to his prolonged dominance, age, and market value , as the Dodgers seem to have an unlimited budget thanks to their TV deal, which allowed for such a record-breaking deal. With Kershaw locked up, the next generational talent name likely to receive an in-house extension within a major market would be Los Angeles Angels’ outfielder Mike Trout.
Trout has posted back-to-back 10-plus WAR seasons to start his career. After earning $500,000 in 2013, the Halos are about to reach arbitration figures that have never been seen before. ESPN’s Dan Szymborski wrote an interesting piece on Wednesday detailing the possible long-term contract that Trout could earn due to his production:
Assuming Trout receives league-minimum salary for 2014 and arbitration awards of roughly 25 percent, 45 percent and 70 percent of his open market value from 2015-17 (superstars tend not to do as well on a percentage basis in arbitration as typical players do), ZiPS estimates $69 million as a fair offer to get Trout through his arbitration years. Then the fun begins.
Even at 7.7 WAR (his 2018 projection as of now), if the value of one WAR increases at 5 percent from the $5.45 million I estimate that teams are paying for this in 2014, that’s enough to get Trout past the $50 million mark per season. So if we are estimating a 10-year deal, that gives us $69 million for his next four years, plus $312 million for the following six seasons (2018-23), for a total of $381 million over 10 years.
Szymborski detailed how it wouldn’t be wise for the Angels to wait much longer on a potential long-term contract, adding:
If Trout plays up to his elite level this season, the cost of signing him for 2018 through 2023 goes up substantially. While we originally calculated that time period to cost $312 million, it goes up to $335 million if he meets his 2014 projection.
If he hits his new 2015 projection, that goes up again to $362 million before 2016. And if he continues to hit his mean projection for the 2017 season, that goes up to $395 million. In other words, if the Angels continued to go year-to-year with Trout and nothing terrible happens in the interim, the price to sign him just for 2018-2023 pops up by more than $80 million.
The Angels could really use Mike Trout for the next decade, especially after locking up Albert Pujols and Josh Hamilton to long-term, lucrative deals, only to see them fail to live up to projections and expectations, but isn’t that the problem with projections?
Teams may have access to unlimited amounts of data, the eyes of scouts, and brilliant baseball minds, but you can’t project injuries; otherwise, Chris Sale‘s left arm would have had a surgery or two on it by now, right? You can’t predict when a player declines a bit too early, as Prince Fielder proved at the age of 29, when he posted an .819 OPS in 2013, the lowest of his professional career, despite Miguel Cabrera‘s continued dominance around him in the Detroit Tigers lineup.
However, teams have long gambled on the skills of players in major sports. It’s why Kwame Brown was employed by seven NBA teams by the age of 30, and why Mark Mulder received a spring training invite after not throwing a pitch in MLB since 2008.
Regardless, sabermetrics continue to influence the front offices across Major League Baseball, as teams continue to use data to develop a better understanding of player values. A lot of times, it seems that those player values, particularly within free agency, are founded upon those educated guesses that I consider gambles. Obviously, the general manager is spending someone else’s money so liberally in these situations, and the incoming revenues lead to a heavy squashing of a lot of the risks involved, but are players truly worth their WAR value?
Lewie Pollis wrote an article at Beyond the Boxscore which detailed the cost of a win via free agency, finding that wins aren’t worth $5 million, as Szymborski stated and what FanGraphs’ Dave Cameron discussed several years ago. Instead, Pollis found data which details a more dramatic cost to improve a club:
So if you’re the owner of an MLB franchise and you want to make your team one win better, you should expect to have to pay $7 million. Planning to bring in a league-average player? That’ll be $14 million. And if you’re willing to splurge to move up 10 games in the standings, you’d better be prepared to open your wallet to the tune of $70 million.
Pollis’ research is much more thorough and includes a lot more information than previous win-value research, but if teams are actually spending $7 million per win from season to season, and that number is only going to increase with the infusion of more money through various media revenue streams, how will “small-market” teams improve or compete in the next few years if they are unable to sign a league-average player for less than $14 million per season?
If projections are essentially educated guesses or hypothesis, then how about this:
Suppose Josh Donaldson was a free agent after the 2013 season. He signs with the Houston Astros, replacing Matt Dominguez (who was worth 1.0 WAR in 2013), which would improve the Astros by approximately six wins, as Donaldson was a 7.7 WAR player in 2013. Based on the cost per win, it would take nearly $42 million per season to sign Donaldson, while improving the Astros from a horrific 51 win team to a horrific 57 win team. Meanwhile, Donaldson, who just turned 28 in December and has all of 996 plate appearances in the majors, becomes the highest paid player in baseball.
Sure, this scenario isn’t playing out this winter because Donaldson isn’t and won’t be a free agent until 2019, but what is the value of win-values when you consider that teams aren’t going to pay players like Donaldson, coming off of a career season after spending five and a half years in the minors, like a superstar? Certainly, clubs bask in the glory of receiving superstar production from their players while they are being underpaid as pre-arbitration or arbitration-eligible major league roster-worthy talent, but is it fair to expect or anticipate clubs spending money based on their value when compared to replacement level talent?
Not everyone in Major League Baseball is replacement level and the fact that WAR is a comparison of an average minor league player with major league talent seems insane. Of course Mike Trout has a 10 WAR when he has had two incredible seasons when compared to your average player – that number should be gigantic when compared to someone who loiters within the minor leagues for several years; however, saying that Josh Donaldson and his 7.7 WAR from 2013, was only worth 2.7 fewer wins than Trout seems kind of insane, as well. Defense and offensive numbers aside, how about a dose of reality?
I love number crunching as much as the next guy, I even spent quite a bit of time trying to create my own value system over the winter, but the idea that WAR is tied to wins and those wins should be tied to free agent contracts, at least in the eyes of sabermetric gurus, seems horribly wrong. While the money may appear endless with all of the new media deals, there will come a point down the road that your mediocre, utility player could be earning $10 million per season – just because the money is there to warrant the contract. Is that good for baseball? Then, when baseball fails because it has become too big for its own britches, what will the gurus crunch for enjoyment?
WAR is great and fun, and it is very creative…but it is flawed. Why else would there be three different theories that provide the same type of data, thanks to Baseball Reference, Baseball Prospectus, and FanGraphs? If the value of a player is based upon a statistic that can be skewed, molded, or shaped based on the philosophy of its creator, is it fair to use that statistic to determine the finances of the game?
I don’t think so. Sabermetrics are great and they allow for a lot of debate, but a lot of that information has become just that…information. Data that can be manipulated like any other form of a statistic.