Project Achoo: A Practical Model and Application for COVID-19 Detection From Recordings of Breath, Voice, and Cough

The COVID-19 pandemic created significant interest and demand for infection detection and monitoring solutions. In this paper, we propose a machine learning method to quickly detect COVID-19 using audio recordings made on consumer devices. The approach combines signal processing and noise removal methods with an ensemble of fine-tuned deep learning networks and enables COVID detection on coughs. We have also developed and deployed a mobile application that uses a symptoms checker together with voice, breath, and cough signals to detect COVID-19 infection. The application showed robust performance on both openly sourced datasets and the noisy data collected during beta testing by the end users.

[1]  Elgar Fleisch,et al.  Towards Device-Agnostic Mobile Cough Detection with Convolutional Neural Networks , 2019, 2019 IEEE International Conference on Healthcare Informatics (ICHI).

[2]  N. Gray,et al.  Is “no test is better than a bad test”? Impact of diagnostic uncertainty in mass testing on the spread of COVID-19 , 2020, medRxiv.

[3]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[4]  Marc Arbyn,et al.  The prevalence of symptoms in 24,410 adults infected by the novel coronavirus (SARS-CoV-2; COVID-19): A systematic review and meta-analysis of 148 studies from 9 countries , 2020, PloS one.

[5]  David Atienza,et al.  The COUGHVID crowdsourcing dataset, a corpus for the study of large-scale cough analysis algorithms , 2021, Scientific data.

[6]  Arsha Nagrani,et al.  Cough Against COVID: Evidence of COVID-19 Signature in Cough Sounds , 2020, ArXiv.

[7]  Xavier Serra,et al.  FSD50K: an Open Dataset of Human-Labeled Sound Events , 2021, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[8]  Mehrin Kiani,et al.  A Generic Deep Learning Based Cough Analysis System From Clinically Validated Samples for Point-of-Need Covid-19 Test and Severity Levels , 2021, IEEE Transactions on Services Computing.

[9]  D. Larremore,et al.  Test sensitivity is secondary to frequency and turnaround time for COVID-19 screening , 2020, Science Advances.

[10]  D. Chicco,et al.  The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation , 2020, BMC Genomics.

[11]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Hasan Farooq,et al.  Can Machine Learning Be Used to Recognize and Diagnose Coughs? , 2020, 2020 International Conference on e-Health and Bioengineering (EHB).

[13]  Sung-Hwan Shin,et al.  Automatic Detection System for Cough Sounds as a Symptom of Abnormal Health Condition , 2009, IEEE Transactions on Information Technology in Biomedicine.

[14]  Md. Milon Islam,et al.  Deep Learning Applications to Combat Novel Coronavirus (COVID-19) Pandemic , 2020, SN Computer Science.

[15]  Sebastian Nowozin,et al.  Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift , 2019, NeurIPS.

[16]  Udantha R. Abeyratne,et al.  Automatic Croup Diagnosis Using Cough Sound Recognition , 2019, IEEE Transactions on Biomedical Engineering.

[17]  Thierry Dutoit,et al.  Assessment of audio features for automatic cough detection , 2011, 2011 19th European Signal Processing Conference.

[18]  Elliot Saba.  Techniques for Cough Sound Analysis , 2018 .

[19]  Cecilia Mascolo,et al.  Exploring Automatic Diagnosis of COVID-19 from Crowdsourced Respiratory Sound Data , 2020, KDD.

[20]  Renard Xaviero Adhi Pramono,et al.  A Cough-Based Algorithm for Automatic Diagnosis of Pertussis , 2016, PloS one.

[21]  Shlomo Berkovsky,et al.  Detecting pertussis in the pediatric population using respiratory sound events and CNN , 2021, Biomed. Signal Process. Control..

[22]  Kofi Odame,et al.  Deep Neural Networks for Identifying Cough Sounds , 2016, IEEE Transactions on Biomedical Circuits and Systems.

[23]  Kun Qian,et al.  An Early Study on Intelligent Analysis of Speech under COVID-19: Severity, Sleep Quality, Fatigue, and Anxiety , 2020, INTERSPEECH.

[24]  Gadi Pinkas,et al.  SARS-CoV-2 Detection From Voice , 2020, IEEE Open Journal of Engineering in Medicine and Biology.

[25]  Vinayak Swarnkar,et al.  Neural network based algorithm for automatic identification of cough sounds , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[26]  Aren Jansen,et al.  CNN architectures for large-scale audio classification , 2016, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[27]  Yusuf A. Amrulloh,et al.  Cough Sound Analysis for Pneumonia and Asthma Classification in Pediatric Population , 2015, 2015 6th International Conference on Intelligent Systems, Modelling and Simulation.

[28]  Muhammad Nabeel,et al.  AI4COVID-19: AI enabled preliminary diagnosis for COVID-19 from cough samples via an app , 2020, Informatics in Medicine Unlocked.

[29]  F. Drobniewski,et al.  False-positive COVID-19 results: hidden problems and costs , 2020, The Lancet Respiratory Medicine.

[30]  P. Whiting,et al.  Interpreting a covid-19 test result , 2020, BMJ.

[31]  Brian Subirana,et al.  COVID-19 Artificial Intelligence Diagnosis Using Only Cough Recordings , 2020, IEEE Open Journal of Engineering in Medicine and Biology.

[32]  Mohammad Nurul Huda,et al.  Cough detection using speech analysis , 2015, 2015 18th International Conference on Computer and Information Technology (ICCIT).

[33]  Andrey Filchenkov,et al.  Noise Masking Recurrent Neural Network for Respiratory Sound Classification , 2018, ICANN.

[34]  Björn Schuller,et al.  COVID-19 detection from audio: seven grains of salt , 2021, The Lancet Digital Health.

[35]  Vinayak Swarnkar,et al.  Wavelet Augmented Cough Analysis for Rapid Childhood Pneumonia Diagnosis , 2015, IEEE Transactions on Biomedical Engineering.

[36]  Paulo Carvalho,et al.  Voluntary cough detection by internal sound analysis , 2014, 2014 7th International Conference on Biomedical Engineering and Informatics.

[37]  Daniel P. W. Ellis,et al.  Learning Sound Event Classifiers from Web Audio with Noisy Labels , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[38]  Alice Baird,et al.  End-to-end convolutional neural network enables COVID-19 detection from breath and cough audio: a pilot study , 2021, BMJ Innovations.

[39]  Frank Knoefel,et al.  Feature extraction for the differentiation of dry and wet cough sounds , 2011, 2011 IEEE International Symposium on Medical Measurements and Applications.

[40]  Tie-Yan Liu,et al.  LightGBM: A Highly Efficient Gradient Boosting Decision Tree , 2017, NIPS.

[41]  Paolo Soda,et al.  An automated and unobtrusive system for cough detection , 2017, 2017 IEEE Life Sciences Conference (LSC).

[42]  Manoranjan Paul,et al.  COVID-19 Control by Computer Vision Approaches: A Survey , 2020, IEEE Access.

[43]  Nicholas Gray,et al.  Is “no test is better than a bad test”? Impact of diagnostic uncertainty in mass testing on the spread of COVID-19 , 2020, PloS one.

[44]  Daniel P. W. Ellis,et al.  General-purpose Tagging of Freesound Audio with AudioSet Labels: Task Description, Dataset, and Baseline , 2018, DCASE.

[45]  David H. Evans,et al.  Detection of cough signals in continuous audio recordings using hidden Markov models , 2006, IEEE Transactions on Biomedical Engineering.

[46]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[47]  M Javorka,et al.  Distinction between voluntary cough sound and speech in volunteers by spectral and complexity analysis. , 2008, Journal of physiology and pharmacology : an official journal of the Polish Physiological Society.

[48]  Srikanth Raj Chetupalli,et al.  Coswara - A Database of Breathing, Cough, and Voice Sounds for COVID-19 Diagnosis , 2020, INTERSPEECH.

[49]  Manvel Avetisian,et al.  CoRSAI: A System for Robust Interpretation of CT Scans of COVID-19 Patients Using Deep Learning , 2021, ACM Trans. Manag. Inf. Syst..

[50]  Karol J. Piczak.  Environmental sound classification with convolutional neural networks , 2015, 2015 IEEE 25th International Workshop on Machine Learning for Signal Processing (MLSP).

[51]  Sriram Chellappan,et al.  On Detecting Chronic Obstructive Pulmonary Disease (COPD) Cough using Audio Signals Recorded from Smart-Phones , 2018, HEALTHINF.

[52]  Esther Rodríguez-Villegas,et al.  Automatic Identification of Cough Events from Acoustic Signals , 2019, 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).