Plus disease in retinopathy of prematurity: qualitative analysis of diagnostic process by experts.

IMPORTANCE Plus disease is the most important parameter that characterizes severe treatment-requiring retinopathy of prematurity, yet diagnostic agreement among experts is imperfect and the precise factors involved in clinical diagnosis are unclear. This study is designed to address these gaps in knowledge by analyzing cognitive aspects of the plus disease diagnostic process by experts. OBJECTIVE To examine the diagnostic reasoning process of experts for plus disease in retinopathy of prematurity using qualitative research techniques. DESIGN Cognitive walk-through, with qualitative analysis of videotaped expert responses and quantitative analysis of expert diagnoses. SETTING Experimental setting in which experts were videotaped while reviewing study data. PARTICIPANTS A panel of international retinopathy of prematurity experts who had the experience of using qualitative retinal features as their primary basis for clinical diagnosis. INTERVENTION Six experts were video recorded while independently reviewing 7 wide-angle retinal images from infants with retinopathy of prematurity. Experts were asked to explain their diagnostic process in detail (think-aloud protocol), mark findings relevant to their reasoning, and diagnose each image (plus vs pre-plus vs neither). Subsequently, each expert viewed the images again while being asked to examine arteries and veins in isolation and answer specific questions. Video recordings were transcribed and reviewed. Diagnostic process of experts was analyzed using a published cognitive model. MAIN OUTCOME AND MEASURES Interexpert and intraexpert agreement. RESULTS Based on the think-aloud protocol, 5 of 6 experts agreed on the same diagnosis in 3 study images and 3 of 6 experts agreed in 3 images. When experts were asked to rank images in order of severity, the mean correlation coefficient between pairs of experts was 0.33 (range, -0.04 to 0.75). All experts considered arterial tortuosity and venous dilation while reviewing each image. Some considered venous tortuosity, arterial dilation, peripheral retinal features, and other factors. When experts were asked to rereview images to diagnose plus disease based strictly on definitions of sufficient arterial tortuosity and venous dilation, all but 1 expert changed their diagnosis compared with the think-aloud protocol. CONCLUSIONS AND RELEVANCE Diagnostic consistency in plus disease is imperfect. Experts differ in their reasoning process, retinal features that they focus on, and interpretations of the same features. Understanding these factors may improve diagnosis and education. Future research defining more precise diagnostic criteria may be warranted.

[1]  McDonnell Tj,et al.  Efficacy of intravitreal bevacizumab for stage 3+ retinopathy of prematurity. , 2011 .

[2]  Michael F Chiang,et al.  Plus disease in retinopathy of prematurity: pilot study of computer-based and expert diagnosis. , 2007, Journal of AAPOS : the official publication of the American Association for Pediatric Ophthalmology and Strabismus.

[3]  Susanne P. Lajoie,et al.  Expert-novice differences in mammogram interpretation , 2007 .

[4]  K. A. Ericsson,et al.  Protocol Analysis: Verbal Reports as Data , 1984 .

[5]  Anna L. Ells,et al.  The International Classification of Retinopathy of Prematurity revisited. , 2005, Archives of ophthalmology.

[6]  Jane S. Myung,et al.  Accuracy of retinopathy of prematurity image-based diagnosis by pediatric ophthalmology fellows: implications for training. , 2011, Journal of AAPOS : the official publication of the American Association for Pediatric Ophthalmology and Strabismus.

[7]  Michael T Trese,et al.  Evidence-based screening criteria for retinopathy of prematurity: natural history data from the CRYO-ROP and LIGHT-ROP studies. , 2002, Archives of ophthalmology.

[8]  Michael F Chiang,et al.  Evaluation of a computer-based system for plus disease diagnosis in retinopathy of prematurity. , 2007, Ophthalmology.

[9]  Birgit Lorenz,et al.  Wide-field digital imaging based telemedicine for screening for acute retinopathy of prematurity (ROP). Six-year results of a multicentre field study , 2009, Graefe's Archive for Clinical and Experimental Ophthalmology.

[10]  Zheen Zhao,et al.  A PILOT STUDY USING ROPtool TO MEASURE RETINAL VASCULAR DILATION , 2009, Retina.

[11]  Michael F Chiang,et al.  ACCURACY OF RETINOPATHY OF PREMATURITY DIAGNOSIS BY RETINAL FELLOWS , 2010, Retina.

[12]  Sin-Ho Jung,et al.  Accuracy of ROPtool vs individual examiners in assessing retinal vascular tortuosity. , 2007, Archives of ophthalmology.

[13]  S. T. Clay,et al.  Computerized analysis of retinal vessel width and tortuosity in premature infants. , 2008, Investigative ophthalmology & visual science.

[14]  Alice Z Chuang,et al.  Efficacy of intravitreal bevacizumab for stage 3+ retinopathy of prematurity. , 2011, The New England journal of medicine.

[15]  Justin Starren,et al.  Telemedical diagnosis of retinopathy of prematurity intraphysician agreement between ophthalmoscopic examination and image-based interpretation. , 2008, Ophthalmology.

[16]  Joseph L. Kannry,et al.  Integrating usability testing and think-aloud protocol analysis with "near-live" clinical simulations in evaluating clinical decision support , 2012, Int. J. Medical Informatics.

[17]  W. Tasman,et al.  Revised indications for the treatment of retinopathy of prematurity: results of the early treatment for retinopathy of prematurity randomized trial. , 2004, Archives of ophthalmology.

[18]  M. E. Martínez-Pérez,et al.  Plus disease in retinopathy of prematurity: development of composite images by quantification of expert opinion. , 2008, Investigative ophthalmology & visual science.

[19]  Gui-shuang Ying,et al.  Image analysis of posterior pole vessels identifies type 1 retinopathy of prematurity. , 2009, Journal of AAPOS : the official publication of the American Association for Pediatric Ophthalmology and Strabismus.

[20]  Jacob Cohen,et al.  Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. , 1968 .

[21]  S. Dai,et al.  Efficacy of wide‐field digital retinal imaging for retinopathy of prematurity screening , 2010, Clinical & experimental ophthalmology.

[22]  Gui-Shuang Ying,et al.  Progression to severe retinopathy predicted by retinal vessel diameter between 31 and 34 weeks of postconception age. , 2007, Archives of ophthalmology.

[23]  Michael J. Prietula,et al.  A Protocol-Based Coding Scheme for the Analysis of Medical Reasoning , 1992, Int. J. Man Mach. Stud..

[24]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[25]  A. Fulton,et al.  Diagnosis of plus disease in retinopathy of prematurity using Retinal Image multiScale Analysis. , 2005, Investigative ophthalmology & visual science.

[26]  Martin A Lindquist,et al.  PLUS DISEASE IN RETINOPATHY OF PREMATURITY: Diagnostic Impact of Field of View , 2011, Retina.

[27]  Yunling E. Du,et al.  Image analysis for retinopathy of prematurity diagnosis. , 2009, Journal of AAPOS : the official publication of the American Association for Pediatric Ophthalmology and Strabismus.

[28]  M. Mills,et al.  Semiautomated analysis of retinal vessel diameter in retinopathy of prematurity patients with and without plus disease. , 2007, American journal of ophthalmology.

[29]  K. Parker,et al.  Semiautomated computer analysis of vessel growth in preterm infants without and with ROP , 2003, The British journal of ophthalmology.

[30]  Justin Starren,et al.  Telemedical retinopathy of prematurity diagnosis: accuracy, reliability, and image quality. , 2007, Archives of ophthalmology.

[31]  M. Morales i Ballús,et al.  RetCam imaging for retinopathy of prematurity screening , 2006 .

[32]  THE PHOTOGRAPHIC SCREENING FOR RETINOPATHY OF PREMATURITY STUDY (PHOTO-ROP): Primary Outcomes , 2008, Retina.

[33]  Michael F Chiang,et al.  Plus disease in retinopathy of prematurity: quantitative analysis of vascular change. , 2010, American journal of ophthalmology.

[34]  E. Lad,et al.  Stanford University network for diagnosis of retinopathy of prematurity (SUNDROP): 36-month experience with telemedicine screening. , 2009, Ophthalmic surgery, lasers & imaging : the official journal of the International Society for Imaging in the Eye.

[35]  W. Fierson Screening Examination of Premature Infants for Retinopathy of Prematurity , 2013, Pediatrics.

[36]  Charles P. Friedman,et al.  Development of visual diagnostic expertise in pathology , 2001, AMIA.

[37]  A R Fielder,et al.  UK retinopathy of prematurity guideline , 2009, Eye.

[38]  M. E. Martínez-Pérez,et al.  Plus disease in retinopathy of prematurity: quantitative analysis of standard published photograph. , 2010, Archives of ophthalmology.

[39]  Michael F Chiang,et al.  Agreement among pediatric ophthalmologists in diagnosing plus and pre-plus disease in retinopathy of prematurity. , 2008, Journal of AAPOS : the official publication of the American Association for Pediatric Ophthalmology and Strabismus.

[40]  D. Wallace,et al.  A pilot study using "ROPtool" to quantify plus disease in retinopathy of prematurity. , 2007, Journal of AAPOS : the official publication of the American Association for Pediatric Ophthalmology and Strabismus.

[41]  A. Fielder,et al.  Preliminary results of treatment of eyes with high-risk prethreshold retinopathy of prematurity in the early treatment for retinopathy of prematurity randomized trial. , 2003, Archives of ophthalmology.

[42]  J. Català Mora,et al.  Computer-automated quantification of plus disease in retinopathy of prematurity , 2003 .

[43]  Michael F Chiang,et al.  Interexpert agreement of plus disease diagnosis in retinopathy of prematurity. , 2007, Archives of ophthalmology.

[44]  L. Brooks,et al.  The development of expertise in dermatology. , 1989, Archives of dermatology.

[45]  Justin Starren,et al.  Accuracy and reliability of remote retinopathy of prematurity diagnosis. , 2006, Archives of ophthalmology.

[46]  John A. Jones,et al.  Multicenter trial of cryotherapy for retinopathy of prematurity: preliminary results. Cryotherapy for Retinopathy of Prematurity Cooperative Group. , 1988, Pediatrics.

[47]  Penny Jennett,et al.  Telemedicine approach to screening for severe retinopathy of prematurity: a pilot study. , 2003, Ophthalmology.