Measurement issues in taxonomic reliability

Work in safety management often involves classification of events using coding schemes or "taxonomies". Such schemes contain separate categories, and users have to reliably choose which codes apply to the events in question. The usefulness of any system is limited by the reliability with which it can be employed, that is the consensus that can be reached on application of codes. This technical note is concerned with practical and theoretical issues in defining and measuring such reliability. Three problem areas are covered: the use of correlational measures, the reporting and calculating of indices of concordance and the use of correction coefficients.