Inter observer agreement about traffic conflicts: a fourth opinion

Oppe has recently published data showing the gradings on a 4 point scale of the severity of 27 traffic conflicts by 10 observers. Three analyses of these data were also reported by various authors. The present paper argues that attention should be given to whether the disagreements between their observers are ascribable to differences in the thresholds between their categories, to errors in construction of the severity scale, or to random inaccuracies; and that simple descriptive methods can take us a long way towards the answer. For the data set considered, about half of the disagreements between observers could be explained by threshold differences (a).