Â
In a previous post to the ISC website (http://tinyurl.com/khwx8y4) I presented an analysis of the reliability and validity of the ISC rankings for the period 2002-2012. The present post is an update that includes the data from 2013. Refer to the previous post for a description of the measures and methodology involved in the analysis. Reliability was measured with Kendall's Coefficient of Concordance (W). Validity was measured with the Spearman Rank Order Correlation Coefficient (rho).
Â
The 2013 ratings replicated the highly satisfactory results shown by the previous analyses. The reliability measure was .897, and the validity measure was .853, the maximum value attainable for each measure being 1.00. The values corresponding to each of the years from 2002 to 2013 are given below. For comparison purposes, I calculated the validity of the published odds for the 19 horses in the 2013 Kentucky Derby in predicting the order of finish for the horses in the field. The Spearman rho value was a meager ..157.
Â
Reliability:Â Kendall 's Coefficient of Concordance (W)
Â
2002Â Â Â Â ..932
2003Â Â Â Â ..897
2004Â Â Â Â ..870
2005Â Â Â Â ..897
2006Â Â Â Â ..901
2007Â Â Â Â ..945
2008Â Â Â Â ..948
2009Â Â Â Â ..927
2010Â Â Â Â ..857
2011Â Â Â Â ..980
2012Â Â Â Â ..949
2013Â Â Â Â ..897
Â
Mean   ..917
Â
Validity: Spearman's Rank Order Correlation Coefficient (rho)
Â
2002Â Â Â Â ..749
2003Â Â Â Â ..909
2004 Â Â Â ..733
2005Â Â Â Â ..600
2006Â Â Â Â ..794
2007Â Â Â Â ..799
2008Â Â Â Â ..748
2009Â Â Â Â ..877
2010Â Â Â Â ..719
2011Â Â Â Â ..790
2012Â Â Â Â ..821
2013Â Â Â Â ..853
Â
Mean = .783
Â
Submitted by Jim Johnson
Vancouver
Â
