Plus disease in retinopathy of prematurity: pilot study of computer-based and expert diagnosis.

PURPOSE To measure accuracy of plus disease diagnosis by recognized experts in retinopathy of prematurity (ROP), and to conduct a pilot study examining performance of a computer-based image analysis system, Retinal Image multiScale Analysis (RISA). METHODS Twenty-two ROP experts independently interpreted a set of 34 wide-angle retinal images for presence of plus disease. A reference standard diagnosis based on expert consensus was defined for each image. Images were analyzed by the computer-based system using individual and linear combinations of system parameters for arterioles and venules: integrated curvature (IC), diameter, and tortuosity index (TI). Sensitivity, specificity, and receiver operating characteristic areas under the curve (AUC) for plus disease diagnosis compared with the reference standard were determined for each expert, as well as for the computer-based system. RESULTS Expert sensitivity ranged from 0.308 to 1.000, specificity ranged from 0.571 to 1.000, and AUC ranged from 0.784 to 1.000. Among individual computer system parameters, venular IC had highest AUC (0.853). Among all computer system parameters, the linear combination of arteriolar IC, arteriolar TI, venular IC, venular diameter, and venular TI had highest AUC (0.967), which was greater than that of 18 (81.8%) of 22 experts. CONCLUSIONS Accuracy of ROP experts for plus disease diagnosis is imperfect. A computer-based image analysis system has potential to diagnose plus disease with high accuracy. Further research involving RISA system parameter cut-off values from this study are required to fully validate performance of this computer-based system compared with that of human experts.

[1]  J A Kors,et al.  The Delphi Method to Validate Diagnostic Knowledge in Computerized ECG Interpretation , 1990, Methods of Information in Medicine.

[2]  D. Wallace,et al.  Prognostic significance of vascular dilation and tortuosity insufficient for plus disease in retinopathy of prematurity. , 2000, Journal of AAPOS : the official publication of the American Association for Pediatric Ophthalmology and Strabismus.

[3]  David W. Bates,et al.  Research Paper: KnowledgeLink: Impact of Context-Sensitive Information Retrieval on Clinicians' Information Needs , 2006, J. Am. Medical Informatics Assoc..

[4]  E. Balas,et al.  Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success , 2005, BMJ : British Medical Journal.

[5]  J. Flynn,et al.  Supplemental Therapeutic Oxygen for Prethreshold Retinopathy Of Prematurity (STOP-ROP), a randomized, controlled trial. I: primary outcomes. , 2000, Pediatrics.

[6]  K. Parker,et al.  Semiautomated computer analysis of vessel growth in preterm infants without and with ROP , 2003, The British journal of ophthalmology.

[7]  Julien Jomier,et al.  Computer-automated quantification of plus disease in retinopathy of prematurity. , 2003, Journal of AAPOS : the official publication of the American Association for Pediatric Ophthalmology and Strabismus.

[8]  J A Hanley,et al.  Sampling variability of nonparametric estimates of the areas under receiver operating characteristic curves: an update. , 1997, Academic radiology.

[9]  J J Capowski,et al.  Observer sensitivity to retinal vessel diameter and tortuosity in retinopathy of prematurity: a model system. , 1996, Journal of pediatric ophthalmology and strabismus.

[10]  B. Munoz,et al.  Blindness and visual impairment in the Americas and the Caribbean , 2002, The British journal of ophthalmology.

[11]  Joseph J. Capowski,et al.  A numeric index based on spatial frequency for the tortuosity of retinal vessels and its application to plus disease in retinopathy of prematurity. , 1995 .

[12]  John A. Jones,et al.  Multicenter trial of cryotherapy for retinopathy of prematurity: preliminary results. Cryotherapy for Retinopathy of Prematurity Cooperative Group. , 1988, Pediatrics.

[13]  A. Fielder,et al.  Preliminary results of treatment of eyes with high-risk prethreshold retinopathy of prematurity in the early treatment for retinopathy of prematurity randomized trial. , 2003, Archives of ophthalmology.

[14]  C. Gilbert,et al.  Childhood blindness in the context of VISION 2020--the right to sight. , 2001, Bulletin of the World Health Organization.

[15]  S. Menzies,et al.  Accuracy of computer diagnosis of melanoma: a quantitative meta-analysis. , 2003, Archives of dermatology.

[16]  N. Ku,et al.  Automated Papanicolaou smear analysis as a screening tool for female lower genital tract malignancies. , 1999, Current opinion in obstetrics & gynecology.

[17]  Michael T Trese,et al.  Evidence-based screening criteria for retinopathy of prematurity: natural history data from the CRYO-ROP and LIGHT-ROP studies. , 2002, Archives of ophthalmology.

[18]  E. Palmer,et al.  15-year outcomes following threshold retinopathy of prematurity: final results from the multicenter trial of cryotherapy for retinopathy of prematurity. , 2005, Archives of ophthalmology.

[19]  Evangelia Micheli-Tzanakou,et al.  Classification of Retinal Damage by a Neural Network Based System , 1998, Journal of Medical Systems.

[20]  H E Rockette,et al.  The use of continuous and discrete confidence judgments in receiver operating characteristic studies of diagnostic imaging techniques. , 1992, Investigative radiology.

[21]  W. Tasman,et al.  Multicenter trial of cryotherapy for retinopathy of prematurity. , 1988, Archives of ophthalmology.

[22]  Anil A. Bharath,et al.  Segmentation of blood vessels from red-free and fluorescein retinal images , 2007, Medical Image Anal..

[23]  Michael F Chiang,et al.  Interexpert agreement of plus disease diagnosis in retinopathy of prematurity. , 2007, Archives of ophthalmology.

[24]  N. Goldschlager,et al.  Status of computerized electrocardiography. , 2006, Cardiology clinics.

[25]  J Starren,et al.  Remote image based retinopathy of prematurity diagnosis: a receiver operating characteristic analysis of accuracy , 2006, British Journal of Ophthalmology.

[26]  Anna L. Ells,et al.  The International Classification of Retinopathy of Prematurity revisited. , 2005, Archives of ophthalmology.

[27]  Justin Starren,et al.  Telemedical retinopathy of prematurity diagnosis: accuracy, reliability, and image quality. , 2007, Archives of ophthalmology.

[28]  A. Patz,et al.  The new international classification of retinopathy of prematurity. , 1984, Archives of ophthalmology.

[29]  Enrique F Schisterman,et al.  On linear combinations of biomarkers to improve diagnostic accuracy , 2005, Statistics in medicine.

[30]  Justin Starren,et al.  Accuracy and reliability of remote retinopathy of prematurity diagnosis. , 2006, Archives of ophthalmology.

[31]  A. Fulton,et al.  Diagnosis of plus disease in retinopathy of prematurity using Retinal Image multiScale Analysis. , 2005, Investigative ophthalmology & visual science.

[32]  Justin Starren,et al.  Telemedical diagnosis of retinopathy of prematurity intraphysician agreement between ophthalmoscopic examination and image-based interpretation. , 2008, Ophthalmology.

[33]  Anil A. Bharath,et al.  Retinal vascular tree morphology: a semi-automatic quantification , 2002, IEEE Transactions on Biomedical Engineering.

[34]  D. Wallace,et al.  A pilot study using "ROPtool" to quantify plus disease in retinopathy of prematurity. , 2007, Journal of AAPOS : the official publication of the American Association for Pediatric Ophthalmology and Strabismus.

[35]  Kevin M. Jackson,et al.  Cost-utility analysis of telemedicine and ophthalmoscopy for retinopathy of prematurity management. , 2008, Archives of ophthalmology.

[36]  Isaac Ben-Sira,et al.  An international classification of retinopathy of prematurity. Clinical experience. , 1985, Ophthalmology.