Experts do not agree when to treat retinopathy of prematurity based on plus disease

Objectives To investigate inter-reader agreement on five severity levels of central vascular changes (none, mild, moderate, severe pre-plus disease, plus disease) and aggressive posterior retinopathy of prematurity (ROP), and to see whether an unintended shift in indication for treatment occurred. Methods Four international ROP readers participated. Before the grading of the photographs, the readers were informed that a high proportion of advanced ROP cases were included. In total, 243 photographs/948 quadrants were available from 136 infants. As a standard series of photographs was available, grading was performed under optimised conditions. Results The four readers agreed on the quadrant scores of only 70 (7.38%) of the 948 quadrants—that is, on 1, 5, 15, 4 and 45 quadrants for scores 0, 1, 2, 3 and 4, respectively. The mean scores differed systematically between the readers (permutation test, p<0.0001). Agreement on presence of aggressive posterior ROP from all four readers was not obtained for any of the photographs. Readers scored plus disease in at least two quadrants in 95.5% of the eyes for which treatment was indicated. All four readers agreed on the scoring of indication for treatment for 195 eyes (80.2%); however, treatment was only recommended in 18 (7.4%) eyes. One reader was found to differ systematically from the others in indicating treatment (Rasch analysis; p=0.0001). Finally, a significant shift in indication for treatment occurred between birth period 2000–2002 and 2003–2006 (Mann–Whitney rank sum test, p<0.001). Conclusions Inter-reader agreement on central vascular changes is poor, especially when based on more than two rating categories. The subjective nature of diagnosing such vascular changes possibly resulted in earlier treatment of preterm infants in Denmark over the entire study period (1997–2006). The recent increased incidence of treated infants in Denmark is, at least in part, explained by a significant shift in indication for treatment.

[1]  Zheen Zhao,et al.  Computer-assisted measurement of retinal vascular width and tortuosity in retinopathy of prematurity. , 2010, Archives of ophthalmology.

[2]  D. Henderson-smart,et al.  Does observer bias contribute to variations in the rate of retinopathy of prematurity between centres? , 2008, Clinical & experimental ophthalmology.

[3]  Michael F Chiang,et al.  Interexpert agreement of plus disease diagnosis in retinopathy of prematurity. , 2007, Archives of ophthalmology.

[4]  D. Wallace,et al.  Prognostic significance of vascular dilation and tortuosity insufficient for plus disease in retinopathy of prematurity. , 2000, Journal of AAPOS : the official publication of the American Association for Pediatric Ophthalmology and Strabismus.

[5]  Anthony C. Davison,et al.  Bootstrap Methods and Their Application , 1998 .

[6]  W. Tasman,et al.  Revised indications for the treatment of retinopathy of prematurity: results of the early treatment for retinopathy of prematurity randomized trial. , 2004, Archives of ophthalmology.

[7]  X. Katz,et al.  ZONE I RETINOPATHY OF PREMATURITY: Clinical Characteristics and Treatment Outcomes , 2006, Retina.

[8]  John A. Jones,et al.  Multicenter trial of cryotherapy for retinopathy of prematurity: preliminary results. Cryotherapy for Retinopathy of Prematurity Cooperative Group. , 1988, Pediatrics.

[9]  A. Fielder,et al.  Preliminary results of treatment of eyes with high-risk prethreshold retinopathy of prematurity in the early treatment for retinopathy of prematurity randomized trial. , 2003, Archives of ophthalmology.

[10]  R A Saunders,et al.  The predictive value of posterior pole vessels in retinopathy of prematurity. , 1995, Journal of pediatric ophthalmology and strabismus.

[11]  Michael F Chiang,et al.  Evaluation of a computer-based system for plus disease diagnosis in retinopathy of prematurity. , 2007, Ophthalmology.

[12]  M. Mills,et al.  Semiautomated analysis of retinal vessel diameter in retinopathy of prematurity patients with and without plus disease. , 2007, American journal of ophthalmology.

[13]  S. T. Clay,et al.  Computerized analysis of retinal vessel width and tortuosity in premature infants. , 2008, Investigative ophthalmology & visual science.

[14]  M. E. Martínez-Pérez,et al.  Plus disease in retinopathy of prematurity: quantitative analysis of standard published photograph. , 2010, Archives of ophthalmology.

[15]  Anna L. Ells,et al.  The International Classification of Retinopathy of Prematurity revisited. , 2005, Archives of ophthalmology.

[16]  I. W. Molenaar,et al.  Rasch models: foundations, recent developments and applications , 1995 .

[17]  H. Fledelius,et al.  Treatment for Retinopathy of Prematurity in Denmark in a Ten-Year Period (1996–2005): Is the Incidence Increasing? , 2008, Pediatrics.

[18]  D B Schaffer,et al.  Prognostic factors in the natural course of retinopathy of prematurity. The Cryotherapy for Retinopathy of Prematurity Cooperative Group. , 1993, Ophthalmology.

[19]  Isaac Ben-Sira,et al.  An international classification of retinopathy of prematurity. , 1984, Archives of ophthalmology.

[20]  Michael T Trese,et al.  Evidence-based screening criteria for retinopathy of prematurity: natural history data from the CRYO-ROP and LIGHT-ROP studies. , 2002, Archives of ophthalmology.

[21]  Lanning B. Kline,et al.  Multicenter trial of cryotherapy for retinopathy of prematurity. Preliminary results. Cryotherapy for Retinopathy of Prematurity Cooperative Group. , 1988, Archives of ophthalmology.

[22]  Julien Jomier,et al.  Computer-automated quantification of plus disease in retinopathy of prematurity. , 2003, Journal of AAPOS : the official publication of the American Association for Pediatric Ophthalmology and Strabismus.

[23]  Georg Rasch,et al.  Probabilistic Models for Some Intelligence and Attainment Tests , 1981, The SAGE Encyclopedia of Research Design.

[24]  Michael F Chiang,et al.  Agreement among pediatric ophthalmologists in diagnosing plus and pre-plus disease in retinopathy of prematurity. , 2008, Journal of AAPOS : the official publication of the American Association for Pediatric Ophthalmology and Strabismus.

[25]  Debashis Kushary,et al.  Bootstrap Methods and Their Application , 2000, Technometrics.

[26]  Isaac Ben-Sira,et al.  An international classification of retinopathy of prematurity. Clinical experience. , 1985, Ophthalmology.

[27]  Michael F Chiang,et al.  Plus disease in retinopathy of prematurity: pilot study of computer-based and expert diagnosis. , 2007, Journal of AAPOS : the official publication of the American Association for Pediatric Ophthalmology and Strabismus.

[28]  M. E. Martínez-Pérez,et al.  Plus disease in retinopathy of prematurity: development of composite images by quantification of expert opinion. , 2008, Investigative ophthalmology & visual science.