Benefits of Independent Double Reading in Digital Mammography: A Theoretical Evaluation of All Possible Pairing Methodologies.

RATIONALE AND OBJECTIVES To establish the efficacy of pairing readers randomly and evaluate the merits of developing optimal pairing methodologies. MATERIALS AND METHODS Sensitivity, specificity, and proportion correct were computed for three different case sets that were independently read by 16 radiologists. Performance of radiologists as single readers was compared to expected double reading performance. We theoretically evaluated all possible pairing methodologies. Bootstrap resampling methods were used for statistical analyses. RESULTS Significant improvements in expected performance for double versus single reading (ie, delta performance) were shown for all performance measures and case-sets (p ≤ .003), with overall delta performance across all theoretically possible pairing schemes (n = 10,395) ranging between .05 and .08. Delta performance for the 20 best pairing schemes was significant (p < .001) and ranged between .07 and .10. Delta performance for 20 random pairing schemes was also significant (p ≤ .003) and ranged between .05 and .08. Delta performance for the 20 worst pairing schemes ranged between .03 and .06, reaching significance in delta proportion correct (p ≤ .021) for all three case-sets and in delta specificity for two case-sets (p ≤ .033) but not for a third case-set (p = .131), and not reaching significance in delta sensitivity for any of the three case-sets (.098 ≥ p ≥ .067). CONCLUSION Significant benefits accrue from double reading, and while random reader pairing achieves most double reading benefits, a strategic pairing approach may maximize the benefits of double reading.

[1]  R. Tibshirani,et al.  An introduction to the bootstrap , 1993 .

[2]  P. Brennan,et al.  Errors in Mammography Cannot be Solved Through Technology Alone , 2018, Asian Pacific journal of cancer prevention : APJCP.

[3]  Harry J de Koning,et al.  Independent double reading of screening mammograms in The Netherlands: effect of arbitration following reader disagreements. , 2004, Radiology.

[4]  M. Eckstein,et al.  The wisdom of crowds for visual search , 2017, Proceedings of the National Academy of Sciences.

[5]  M. Wallis,et al.  Changing case Order to Optimise patterns of Performance in mammography Screening (CO-OPS): study protocol for a randomized controlled trial , 2014, Trials.

[6]  K. Zou,et al.  Statistical combination schemes of repeated diagnostic test data. , 2006, Academic radiology.

[7]  Paul Taylor,et al.  Computer aids and human second reading as interventions in screening mammography: two systematic reviews to compare effects on cancer detection and recall rate. , 2008, European journal of cancer.

[8]  E. Thurfjell,et al.  Benefit of independent double reading in a population-based mammography screening program. , 1994, Radiology.

[9]  Montserrat Rué,et al.  Cost-Effectiveness of Double Reading versus Single Reading of Mammograms in a Breast Cancer Screening Programme , 2016, PloS one.

[10]  S. Ciatto,et al.  Second reading of screening mammograms increases cancer detection and recall rates. Results in the Florence screening programme , 2005, Journal of medical screening.

[11]  B. Muir,et al.  The efficacy of double reading mammograms in breast screening. , 1994, Clinical radiology.

[12]  S. Taylor-Phillips,et al.  Double reading in breast cancer screening: considerations for policy-making. , 2020, The British journal of radiology.

[13]  Stefan M. Herzog,et al.  Boosting medical diagnostics by pooling independent judgments , 2016, Proceedings of the National Academy of Sciences.

[14]  Sophia Zackrisson,et al.  Is single reading with computer-aided detection (CAD) as good as double reading in mammography screening? A systematic review , 2012, BMC Medical Imaging.

[15]  Aileen Clarke,et al.  Effect of Using the Same vs Different Order for Second Readings of Screening Mammograms on Rates of Breast Cancer Detection: A Randomized Clinical Trial. , 2016, JAMA.

[16]  Francesca Caumo,et al.  Breast screening using 2D-mammography or integrating digital breast tomosynthesis (3D-mammography) for single-reading or double-reading--evidence to guide future screening strategies. , 2014, European journal of cancer.

[17]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[18]  M. Gromet.  Comparison of computer-aided detection to double reading of screening mammograms: review of 231,221 mammograms. , 2008, AJR. American journal of roentgenology.