Ranking Comparison: WAR and Cy Young Voting
In the last column I introduced a metric to quantify the amount a pair of ranked listed agree. The example worked up two lists of American League (AL) starting pitchers, comparing the top ten AL starters by ERA and WHIP.
The ranking comparison metric doesn't have a lot of power without a lot of data. The lot of data here will be two sets of ranked lists. Going back to 1977, consider AL Cy Young voting and Wins Above Replacemt (WAR) for AL pitchers. The Cy Young award is meant to reward the best pitching performance in each league each season. The award is by vote. Every year the vote recipients are ranked. Pitchers can also be ranked by WAR, a widely agreed upon measure of how much players contribute to team success. In an ideal world, the Cy Young award would go to pitchers who helped their teams the most. This doesn't happen. The metric defined earlier describes how much this doesn't happen.
The metric generates scores ranging from zero to one, one being complete agreement, zero total disagreement. The chart below plots annual agreement of the top five pitchers in AL Cy Young voting since 1977 against the top five AL pitchers by WAR. The AL expanded to fourteen teams in 1977, making the total number of AL pitchers relatively constant during the years in review.
The chart marks year on the horizontal axis, rank agreement score on the vertical axis.
Click image for larger version.
No trend appears here. The bad years, low score years, are those in which relief pitchers got Cy Young attention. It should also be noted that in two of the yeas covered only four pitchers received votes. Guess those years in the comments if you like. The metric still applies though. The only real fudge factor comes up in 1990. Dave Stieb and Dennis Eckersley tied for fifth in Cy Young voting that year. Dave Stieb was a top-five WAR pitcher that year, the Eck wasn't. I calculated the agreement score with Stieb as the fifth place Cy Younger and Eck as the same then averaged the results.
Click image for larger version.
The chart above carries a couple observable trends. Firstly, it trends up. Secondly, it appears somewhat sinusoidal. The second trend I can't explain. The chart is plotted with rolling five year averages of the rank comparison scores. Each vertical axis value represents the average of the year in question, on the horizontal axis, along with the two succeeding and two preceding values. Thus 2011 is the average of 2009-2013 rank comparison values. This chart shows that Cy Young voters are generally improving their recognition of pitcher value.
The equation in the chart defines the trend line. The R-squared value measures the quality of the fit. Roughly, it states that 58 percent of the change in the vertical axis is explained by change in the horizontal axis. This might indicate Cy Young voters are voting according to pitcher contributions more now than 30 years ago, given large enough samples to detect the trend.
The two years with only four AL pitcher recieving Cy Young Award votes were not terribly high scoring years in terms of rank comparison with WAR, which is a little surprising. Obvious Cy Young choices don't match obvious contributions to victory.