Can you believe my eyes? The importance of interobserver reliability statistics in observations of animal behaviour

Interobserver (or inter-rater) reliability is a vital part of all psychological studies that use an observational methodology to address questions of human behaviour. Concerns about reliability in these studies have long since left the arena of ‘should we use an interobserver reliability statistic?’ for debate on the particular type of statistic to be used, and academic careers have been built on this question. In stark contrast, however, it appears to be extremely rare to see interobserver reliability addressed at all in observational studies of animal behaviour. While we would never claim that this omission would or should be a basis for deeming a paper unacceptable, or disregarding its conclusions, we do feel that observational procedures are an integral part of the methodology of many studies, and that their inclusion in published papers should be commonplace. As an informal measure of the frequency with which interobserver reliability was addressed in papers involving the observation of animal behaviour, we surveyed articles recently published in the journal Animal Behaviour. Our data came from volume 75 (3, 4) and volume 76 (1) of the journal, which were at the time the most recent issues available. We examined the first 100 articles (alphabetical by first author) that were methodologically relevant. Articles included in the survey used observational methodologies such as classification of behaviours, judgment of occurrence (or nonoccurrence) of behaviours, and the counting of instances of behaviour. Articles deemed not methodologically relevant and thus excluded from the analysis included studies using computer modelling techniques, studies in which results were strictly nominal (i.e. presence or absence of a physical object or the number of objects present), and studies that dealt with measurable quantitative variables such as weight, length or hormone levels. Studies such as these, of course, are also subject to error on the part of a single experimenter and are always improved by multiple, reliable experimenters; however, the problem is less pressing than it is in studies that deal with strictly behavioural observations. Ninety-six of these 100 articles did not address interobserver reliability in their published text. Of these 96 articles, three mentioned some form of replication of the observations, seven * Correspondence: A. B. Kaufman, Department of Neuroscience, LSP 2915, University of California, Riverside, Riverside, CA 92521, U.S.A. E-mail address: allison.kaufman@email.ucr.edu (A.B. Kaufman). 1 R. Rosenthal is at the Department of Psychology, 3111B Psychology Building, University of California, Riverside, Riverside, CA 92521, U.S.A. E-mail: robert. rosenthal@ucr.edu