Interrater reliability with SPSS for Windows 5.0

Abstract Simple procedures for estimating interrater reliability are presented using the new SPSS for Windows 5.0 program. Both the interrater reliability for averaged ratings and the intraclass correlation for a single, typical judge are derived from the repeated-measures ANOVA output in the RELIABILITY program. The relationship between these two reliability estimates and the Spearman-Brown prophecy formula is also demonstrated. A second computational technique, using RELIABILITY and the FLIP procedure to perform a matrix transposition on the data, estimates interrater reliability for averaged ratings directly and also identifies poorer judges (less consistent with a group).