Assessment of observers' stability and reliability — A tool for evaluation of intra- and inter-concordance in animal behavioral recordings

Behavior studies on the neurobiological effects of environmental, pharmacological and physiological manipulations in lab animals try to correlate these procedures with specific changes in animal behavior. Parameters such as duration, latency and frequency are assessed from the visually recorded sequences of behaviors, to distinguish changes due to manipulation. Since behavioral recording procedure is intrinsically interpretative, high variability in experimental results is expected and usual, due to observer-related influences such as experience, knowledge, stress, fatigue and personal biases. Here, we present a computer program that supports the assessment of inter- and intra-observer concordance, using statistical indices (e.g., Kappa and Kendal coefficients and concordance index). The software was tested in a case study with 4 different observers, naïve to behavioral recording procedures. On paired analysis, the higher agreement index achieved was 0.76 (concordance index) and 0.47 (Kappa Coefficient, where 0 is no agreement and 1 is total agreement). Observers showed poor concordance indices (lower than 0.7), emphasizing the concern on observer recording stability and on precise morphological definition of the recorded behaviors. These indices can also be used to train observers and to refine the behavioral catalogue definitions, as they are related to different behavioral recording aspects.