Interobserver variation in interpreting chest radiographs for the diagnosis of acute respiratory distress syndrome.

To measure the reliability of chest radiographic diagnosis of acute respiratory distress syndrome (ARDS) we conducted an observer agreement study in which two of eight intensivists and a radiologist, blinded to one another's interpretation, reviewed 778 radiographs from 99 critically ill patients. One intensivist and a radiologist participated in pilot training. Raters made a global rating of the presence of ARDS on the basis of diffuse bilateral infiltrates. We assessed interobserver agreement in a pairwise fashion. For rater pairings in which one rater had not participated in the consensus process we found moderate levels of raw (0.68 to 0.80), chance-corrected (kappa 0.38 to 0.55), and chance-independent (Phi 0. 53 to 0.75) agreement. The pair of raters who participated in consensus training achieved excellent to almost perfect raw (0.88 to 0.94), chance-corrected (kappa 0.72 to 0.88), and chance-independent (Phi 0.74 to 0.89) agreement. We conclude that intensivists without formal consensus training can achieve moderate levels of agreement. Consensus training is necessary to achieve the substantial or almost perfect levels of agreement optimal for the conduct of clinical trials.

[1]  M. Lamy,et al.  The American-European Consensus Conference on ARDS. Definitions, mechanisms, relevant outcomes, and clinical trial coordination. , 1994, American journal of respiratory and critical care medicine.

[2]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[3]  Jacob Cohen,et al.  Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. , 1968 .

[4]  Stephen E. Lapinsky,et al.  Evaluation of a Ventilation Strategy to Prevent Barotrauma in Patients at High Risk for Acute Respiratory Distress Syndrome , 1999 .

[5]  W. Willett,et al.  Misinterpretation and misuse of the kappa statistic. , 1987, American journal of epidemiology.

[6]  J. Yelle,et al.  Adult respiratory distress syndrome: A systematic overview of incidence and risk factors , 1996 .

[7]  G H Guyatt,et al.  Interobserver variation in the computed tomographic evaluation of mediastinal lymph node size in patients with potentially resectable lung cancer. Canadian Lung Oncology Group. , 1995, Chest.

[8]  P. Herman,et al.  Interobserver agreement using computed radiography in the adult intensive care unit. , 1994, Academic radiology.

[9]  R. G. Fraser,et al.  DIAGNOSIS OF DISEASES OF THE CHEST , 1978, The Ulster Medical Journal.

[10]  M. Schluchter,et al.  Chest radiographic data acquisition and quality assurance in multicenter studies , 1997, Pediatric Radiology.

[11]  L. Nelson,et al.  High-level positive end-expiratory pressure management in trauma-associated adult respiratory distress syndrome. , 1991, The Journal of trauma.

[12]  N. Breslow,et al.  Statistical methods in cancer research. Vol. 1. The analysis of case-control studies. , 1981 .

[13]  V. Farewell,et al.  Conditional inference for subject-specific and marginal agreement: Two families of agreement measures† , 1995 .

[14]  N Taub,et al.  An assessment of inter-observer agreement and accuracy when reporting plain radiographs. , 1997, Clinical radiology.

[15]  Arthur S Slutsky,et al.  Evaluation of a ventilation strategy to prevent barotrauma in patients at high risk for acute respiratory distress syndrome. Pressure- and Volume-Limited Ventilation Strategy Group. , 1998, The New England journal of medicine.

[16]  J. D. Smith,et al.  Multiple organ system failure and infection in adult respiratory distress syndrome. , 1983, Annals of internal medicine.

[17]  A. Jackson,et al.  Interobserver variation in the chest radiograph component of the lung injury score , 1995, Anaesthesia.

[18]  H. Winer-Muram,et al.  Guidelines for reading and interpreting chest radiographs in patients receiving mechanical ventilation. , 1992, Chest.

[19]  F. Bloomfield,et al.  Inter- and intra-observer variability in the assessment of atelectasis and consolidation in neonatal chest radiographs , 1999, Pediatric Radiology.

[20]  M. Lamy,et al.  Report of the American-European Consensus conference on acute respiratory distress syndrome: definitions, mechanisms, relevant outcomes, and clinical trial coordination. Consensus Committee. , 1994, Journal of critical care.

[21]  J. Fleiss.  Measuring nominal scale agreement among many raters. , 1971 .

[22]  Gordon H. Guyatt,et al.  Clinical Investigations: Imaging: ArticlesInterobserver Variation in the Computed Tomographic Evaluation of Mediastinal Lymph Node Size in Patients With Potentially Resectable Lung Cancer , 1995 .

[23]  J F Murray,et al.  An expanded definition of the adult respiratory distress syndrome. , 1988, The American review of respiratory disease.