Closing the loop for AI-ready radiology

Abstract Background  In recent years, AI has made significant advancements in medical diagnosis and prognosis. However, the incorporation of AI into clinical practice is still challenging and under-appreciated. We aim to demonstrate a possible vertical integration approach to close the loop for AI-ready radiology. Method  This study highlights the importance of two-way communication for AI-assisted radiology. As a key part of the methodology, it demonstrates the integration of AI systems into clinical practice with structured reports and AI visualization, giving more insight into the AI system. By integrating cooperative lifelong learning into the AI system, we ensure the long-term effectiveness of the AI system, while keeping the radiologist in the loop.  Results  We demonstrate the use of lifelong learning for AI systems by incorporating AI visualization and structured reports. We evaluate Memory Aware-Synapses and Rehearsal approach and find that both approaches work in practice. Furthermore, we see the advantage of lifelong learning algorithms that do not require the storing or maintaining of samples from previous datasets. Conclusion  In conclusion, incorporating AI into the clinical routine of radiology requires a two-way communication approach and seamless integration of the AI system, which we achieve with structured reports and visualization of the insight gained by the model. Closing the loop for radiology leads to successful integration, enabling lifelong learning for the AI system, which is crucial for sustainable long-term performance. Key Points:   The integration of AI systems into the clinical routine with structured reports and AI visualization. Two-way communication between AI and radiologists is necessary to enable AI that keeps the radiologist in the loop. Closing the loop enables lifelong learning, which is crucial for long-term, high-performing AI in radiology.

[1]  A. Aisen,et al.  Accelerating Development and Clinical Deployment of Diagnostic Imaging Artificial Intelligence. , 2021, Journal of the American College of Radiology : JACR.

[2]  J. Ross,et al.  Clinical studies sponsored by digital health companies participating in the FDA’s Precertification Pilot Program: A cross-sectional analysis , 2021, Clinical trials.

[3]  K. Kersting,et al.  CLEVA-Compass: A Continual Learning EValuation Assessment Compass to Promote Research Transparency and Comparability , 2021, ICLR.

[4]  Oleg S. Pianykh,et al.  Dynamic memory to alleviate catastrophic forgetting in continual learning with medical imaging , 2021, Nature Communications.

[5]  Anirban Mukhopadhyay,et al.  How Reliable Are Out-of-Distribution Generalization Methods for Medical Image Segmentation? , 2021, GCPR.

[6]  Kerstin N. Vokinger,et al.  Regulating AI in medicine in the United States and Europe , 2021, Nature Machine Intelligence.

[7]  M. Field,et al.  Deep learning for segmentation in radiation therapy planning: a review , 2021, Journal of medical imaging and radiation oncology.

[8]  Anirban Mukhopadhyay,et al.  Detecting when pre-trained nnU-Net models fail silently for Covid-19 lung lesion segmentation , 2021, MICCAI.

[9]  Khurram Khurshid,et al.  Role of deep learning in brain tumor detection and classification (2015 to 2020): A review , 2021, Comput. Medical Imaging Graph..

[10]  A. Kesselheim,et al.  Continual learning in medical devices: FDA's action plan and beyond. , 2021, The Lancet. Digital health.

[11]  Armin Heinzl,et al.  Augmenting Medical Diagnosis Decisions? An Investigation into Physicians' Decision-Making Process with Artificial Intelligence , 2021, Inf. Syst. Res..

[12]  Robyn L. Ball,et al.  The RSNA Pulmonary Embolism CT Dataset. , 2021, Radiology. Artificial intelligence.

[13]  Aneta Lisowska,et al.  Continual Class Incremental Learning for CT Thoracic Segmentation , 2020, DART/DCL@MICCAI.

[14]  Jerry L Prince,et al.  A Review of Deep Learning in Medical Imaging: Imaging Traits, Technology Trends, Case Studies With Progress Highlights, and Future Promises , 2020, Proceedings of the IEEE.

[15]  N. Arun,et al.  Assessing the (Un)Trustworthiness of Saliency Maps for Localizing Abnormalities in Medical Imaging , 2020, medRxiv.

[16]  Wray L. Buntine,et al.  Hands-On Bayesian Neural Networks—A Tutorial for Deep Learning Users , 2020, IEEE Computational Intelligence Magazine.

[17]  Anirban Mukhopadhyay,et al.  M3d-CAM: A PyTorch library to generate 3D data attention maps for medical deep learning , 2020, Bildverarbeitung für die Medizin.

[18]  Erik Ziegler,et al.  Open Health Imaging Foundation Viewer: An Extensible Open-Source Framework for Building Web-Based Imaging Applications to Support Cancer Research , 2020, JCO clinical cancer informatics.

[19]  J. Hatherley.  Limits of trust in medical AI , 2020, Journal of Medical Ethics.

[20]  Enrico Costanza,et al.  Evaluating saliency map explanations for convolutional neural networks: a user study , 2020, IUI.

[21]  Beomsu Kim,et al.  Why are Saliency Maps Noisy? Cause of and Solution to Noisy Saliency Maps , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[22]  David Danks,et al.  Impacts on Trust of Healthcare AI , 2018, AIES.

[23]  Been Kim,et al.  Sanity Checks for Saliency Maps , 2018, NeurIPS.

[24]  Marcus Rohrbach,et al.  Memory Aware Synapses: Learning what (not) to forget , 2017, ECCV.

[25]  Anirban Sarkar,et al.  Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks , 2017, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[26]  Gang Sun,et al.  Squeeze-and-Excitation Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[27]  Felix G. Meinel,et al.  Structured reporting of CT examinations in acute pulmonary embolism. , 2017, Journal of cardiovascular computed tomography.

[28]  Christoph H. Lampert,et al.  iCaRL: Incremental Classifier and Representation Learning , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[30]  S. Hochreiter,et al.  Long Short-Term Memory , 1997, Neural Computation.

[31]  M. Kalra,et al.  Overlooked Trustworthiness of Saliency Maps , 2022, MICCAI.

[32]  A. Mukhopadhyay,et al.  Practical uncertainty quantification for brain tumor segmentation , 2022, MIDL.

[33]  E. Pedroni,et al.  The calibration of CT Hounsfield units for radiotherapy treatment planning. , 1996, Physics in medicine and biology.