Reducing Risk and Uncertainty of Deep Neural Networks on Diagnosing COVID-19 Infection

Effective and reliable screening of patients via Computer-Aided Diagnosis can play a crucial part in the battle against COVID-19. Most of the existing works focus on developing sophisticated methods yielding high detection performance, yet not addressing the issue of predictive uncertainty. In this work, we introduce uncertainty estimation to detect confusing cases for expert referral to address the unreliability of state-of-the-art (SOTA) DNNs on COVID-19 detection. To the best of our knowledge, we are the first to address this issue on the COVID-19 detection problem. In this work, we investigate a number of SOTA uncertainty estimation methods on publicly available COVID dataset and present our experimental findings. In collaboration with medical professionals, we further validate the results to ensure the viability of the best performing method in clinical practice.

[1]  K. Doi.  Overview on research and development of computer-aided diagnostic schemes. , 2004, Seminars in ultrasound, CT, and MR.

[2]  M. Jones,et al.  The role of chest radiography in confirming covid-19 pneumonia , 2020, BMJ.

[3]  Allan Tucker,et al.  Estimating Uncertainty and Interpretability in Deep Learning for Coronavirus (COVID-19) Detection , 2020, ArXiv.

[4]  Evangelos Triantaphyllou,et al.  The seven key challenges for the future of computer-aided diagnosis in medicine , 2019, Int. J. Medical Informatics.

[5]  Amit Sethi,et al.  Functional Space Variational Inference for Uncertainty Estimation in Computer Aided Diagnosis , 2020, ArXiv.

[6]  Prabira Kumar Sethy,et al.  Detection of Coronavirus Disease (COVID-19) Based on Deep Features , 2020 .

[7]  Saeid Belkasim,et al.  A Unified Plug-and-Play Framework for Effective Data Denoising and Robust Abstention , 2020, ArXiv.

[8]  Dinggang Shen,et al.  Machine Learning in Medical Imaging , 2012, Lecture Notes in Computer Science.

[9]  Philip S. Yu,et al.  A Survey on Applications of Artificial Intelligence in Fighting Against COVID-19 , 2020, ACM Comput. Surv..

[10]  Peter L. Bartlett,et al.  Classification with a Reject Option using a Hinge Loss , 2008, J. Mach. Learn. Res..

[11]  C. Messa,et al.  Artificial intelligence applied on chest X-ray can aid in the diagnosis of COVID-19 infection: a first experience from Lombardy, Italy , 2020, medRxiv.

[12]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[13]  Alexander Wong,et al.  COVID-Net: a tailored deep convolutional neural network design for detection of COVID-19 cases from chest X-ray images , 2020, Scientific reports.

[14]  Max-Heinrich Laves,et al.  Uncertainty Quantification in Computer-Aided Diagnosis: Make Your Model say "I don't know" for Ambiguous Cases , 2019, ArXiv.

[15]  Siegfried Wahl,et al.  Leveraging uncertainty information from deep neural networks for disease detection , 2016, Scientific Reports.

[16]  Mario Vento,et al.  A method for improving classification reliability of multilayer perceptrons , 1995, IEEE Trans. Neural Networks.

[17]  Ran El-Yaniv,et al.  SelectiveNet: A Deep Neural Network with an Integrated Reject Option , 2019, ICML.

[18]  Mario Vento,et al.  To reject or not to reject: that is the question-an answer in case of neural classifiers , 2000, IEEE Trans. Syst. Man Cybern. Part C.

[19]  Ruslan Salakhutdinov,et al.  Deep Gamblers: Learning to Abstain with Portfolio Theory , 2019, NeurIPS.

[20]  Chunhua Shen,et al.  COVID-19 Screening on Chest X-ray Images Using Deep Learning based Anomaly Detection , 2020, ArXiv.

[21]  Bhavya Kailkhura,et al.  Probabilistic Neighbourhood Component Analysis: Sample Efficient Uncertainty Estimation in Deep Learning , 2020, ArXiv.

[22]  Philipp Berens,et al.  Expert-validated estimation of diagnostic uncertainty for deep neural networks in diabetic retinopathy detection , 2019, Medical Image Anal..

[23]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .