On Wednesday, Baseball America released an online version of its annual organizational talent rankings, listings of each major league franchise in order of “the total worth of each farm system’s prospects.” This year, the Pittsburgh Pirates came in first.
I wrote about the expected wins above replacement of individual players in Baseball America’s Top 100 Prospects list, and I wanted to continue the discussion by looking at how the publication’s assessments of organizational talent track with each team’s top prospects’ expected WAR and with future team performance.
Using various sources — including, in large part, this archival Baseball America page — I compiled Baseball America’s organizational lists for every season back to 1984. For years since 1990, we can compare these overall system rankings with those that would be implied by adding up the expected WAR values of each team’s best prospects (those who cracked Baseball America’s Top 100 list).
For example, at the height of their dynasty years in 2000, the New York Yankees were rated No. 1 in Baseball America’s organizational talent rankings. But adding up the expected WAR totals for the Yankees’ Top 100 list members gets us 28.8 — only good for fifth at the time and miles behind the league-leading Marlins’ 51.3 expected WAR. (That Marlins prospect class produced a trio of future ace pitchers in Josh Beckett, A.J. Burnett, and Brad Penny — not a bad haul.)
I’m picking on the Yankees here a bit for having an overrated class of phenoms under the championship halo of 2000 (aside from No. 5 Nick Johnson and No. 16 Alfonso Soriano, their only other Top 100 selections that year were Wily Mo Pena at No. 88 and D’Angelo Jimenez at No. 89). But overall, there’s a strong correlation — a Spearman’s rho of 0.86 — between a team’s Baseball America organizational ranking and its ranking if we just add up the expected WAR values of its players in the Top 100 list.
This year’s top Pirates prospects total an expected WAR of 42.4, which ranks third behind the Chicago Cubs’ 50.3 mark. That’s not quite as wide a gap as the 2000 Yankees. (It’s also worth noting that the distribution of prospects has a long tail, so while the presence of an organization’s farmhands in the Top 100 matters a lot, it’s not everything. It’s completely plausible that a system’s depth beyond the Top 100 can make up several spots in the overall rankings.)
The real question, though, is what this means for the next decade or so of baseball in Pittsburgh. Optimistic Pirates fans might point to the 1992 Atlanta Braves’ top-rated prospect class, which preceded a decade in which the Braves boasted the best record in baseball. But pessimists will quickly find a counterpoint close to home: The 1997 Pirates also had Baseball America’s best crop of prospects, only to have MLB’s third-worst record over the next 10 years.
Anecdotes aside, the general relationship appears to be a weak one. The Spearman rank correlation coefficient between a team’s Baseball America organizational ranking and its rank among MLB teams in winning percentage over the next 10 seasons is just 0.24. However, there are also a lot of other factors at play in a team’s performance beyond just its prospect classes.
Writing at Baseball Analysts in 2010, Sky Andrecheck — currently in the Cleveland Indians’ front office — looked at the ability of Baseball America’s farm rankings to predict future team wins after controlling for existing team quality, team payroll and market size. He found that a team’s running four-year organizational ranking was a significant predictor of a team’s success, and that the top farm system could contribute about 67 more wins than the worst system over the following nine seasons.
Two years later, Matt Swartz of FanGraphs attacked the question from a different direction, measuring a team’s return on investment in terms of “nonmarket WAR” — that is, WAR for which a team does not have to pay the auction-style premiums found on the free agency market (including the dreaded winner’s curse). He found an effect similar to what Andrecheck observed: The best farm system was worth 41 more nonmarket WAR than the worst farm system over the subsequent five seasons.
But Swartz also underscored the risk involved in banking on prospects. A team’s organizational ranking didn’t explain anywhere near as much of its subsequent nonmarket WAR as its payroll explained its future auction-market WAR. Or, as Swartz wrote, “Payroll … can tell you about four times as much as about the difference between teams over the next five years as farm-system rankings can.”
This isn’t great news for the Pirates, who enter the 2014 season with the majors’ fourth-lowest payroll. (That’s not an abnormal ranking for them, either — 2013 was the only season since 2003 in which Pittsburgh didn’t have one of baseball’s 10 lowest payrolls.) For its long-term future, the club is probably banking on the “low payroll/good prospects” model to bring it success. But the research says that path is a risky one, even if a team has a shiny No. 1 Baseball America organizational ranking.