Automatic Non-Invasive Cough Detection based on Accelerometer and Audio Signals

We present an automatic non-invasive way of detecting cough events based on both accelerometer and audio signals. The acceleration signals are captured by a smartphone firmly attached to the patient’s bed, using its integrated accelerometer. The audio signals are captured simultaneously by the same smartphone using an external microphone. We have compiled a manually-annotated dataset containing such simultaneouslycaptured acceleration and audio signals for approximately 6000 cough and 68000 non-cough events from 14 adult male patients in a tuberculosis clinic. LR, SVM and MLP are evaluated as baseline classifiers and compared with deep architectures such as CNN, LSTM, and Resnet50 using a leave-oneout cross-validation scheme. We find that the studied classifiers can use either acceleration or audio signals to distinguish between coughing and other activities including sneezing, throatclearing, and movement on the bed with high accuracy. However, in all cases, the deep neural networks outperform the shallow classifiers by a clear margin and the Resnet50 offers the best performance by achieving an AUC exceeding 0.98 and 0.99 for acceleration and audio signals respectively. While audio-based classification consistently offers a better performance than acceleration-based classification, we observe that the difference is very small for the best systems. Since the acceleration signal requires less processing power, and since the need to record audio is sidestepped and thus privacy is inherently secured, and since the recording device is attached to the bed and not worn, an accelerometer-based highly accurate noninvasive cough detector may represent a more convenient and readily accepted method in long-term cough monitoring.

[1]  Jie Ma,et al.  A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. , 2019, Journal of clinical epidemiology.

[2]  L. T. DeCarlo On the meaning and use of kurtosis. , 1997 .

[3]  Tom Chau,et al.  Automatic discrimination between cough and non-cough accelerometry signal artefacts , 2019, Biomed. Signal Process. Control..

[4]  Hiroshi Yamashita,et al.  An Interior Point Method with a Primal-Dual Quadratic Barrier Penalty Function for Nonlinear Optimization , 2003, SIAM J. Optim..

[5]  Xiaofan Jiang,et al.  PAMS: Improving Privacy in Audio-Based Mobile Systems , 2020, AIChallengeIoT@SenSys.

[6]  Sriram Chellappan,et al.  TussisWatch: A Smart-Phone System to Identify Cough Episodes as Early Symptoms of Chronic Obstructive Pulmonary Disease and Congestive Heart Failure , 2019, IEEE Journal of Biomedical and Health Informatics.

[7]  竹内 一郎,et al.  Leave-One-Out Cross-Validation , 2014, Encyclopedia of Machine Learning and Data Mining.

[8]  N. Wiener The Wiener RMS (Root Mean Square) Error Criterion in Filter Design and Prediction , 1949 .

[9]  T. Sejnowski,et al.  Estimating alertness from the EEG power spectrum , 1997, IEEE Transactions on Biomedical Engineering.

[10]  Jilong Kuang,et al.  A method for preserving privacy during audio recordings by filtering speech , 2017, 2017 IEEE Life Sciences Conference (LSC).

[11]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[12]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[13]  Thierry Dutoit,et al.  Objective Study of Sensor Relevance for Automatic Cough Detection , 2013, IEEE Journal of Biomedical and Health Informatics.

[14]  Yanbin Yuan,et al.  Wind power prediction using hybrid autoregressive fractionally integrated moving average and least square support vector machine , 2017 .

[15]  Peter Wittenburg,et al.  ELAN: a Professional Framework for Multimodality Research , 2006, LREC.

[16]  Alex Sherstinsky,et al.  Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network , 2018, Physica D: Nonlinear Phenomena.

[17]  Oliver Chiu-sing Choy,et al.  An efficient MFCC extraction method in speech recognition , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[18]  Tareq Abed Mohammed,et al.  Understanding of a convolutional neural network , 2017, 2017 International Conference on Engineering and Technology (ICET).

[19]  José J. López,et al.  Cough monitoring for pulmonary tuberculosis using combined microphone/accelerometer measurements , 2014 .

[20]  Ah Chung Tsoi,et al.  Face recognition: a convolutional neural-network approach , 1997, IEEE Trans. Neural Networks.

[21]  Orhan Arikan,et al.  Short-time Fourier transform: two fundamental properties and an optimal implementation , 2003, IEEE Trans. Signal Process..

[22]  Thomas Niesler,et al.  COVID-19 detection in cough, breath and speech using deep transfer learning and bottleneck features , 2021, Computers in Biology and Medicine.

[23]  H. Mochizuki,et al.  A new method for objectively evaluating childhood nocturnal cough , 2015, Pediatric pulmonology.

[24]  Mohamed Moustafa Azmy Feature extraction of heart sounds using velocity and acceleration of MFCCs based on support vector machines , 2017, 2017 IEEE Jordan Conference on Applied Electrical Engineering and Computing Technologies (AEECT).

[25]  R. K. Sinha Artificial neural network detects changes in electro-encephalogram power spectrum of different sleep-wake states in an animal model of heat stress , 2003, Medical and Biological Engineering and Computing.

[26]  R. Martinez-Duarte,et al.  Assessing the importance of the root mean square (RMS) value of different waveforms to determine the strength of a dielectrophoresis trapping force , 2017, Electrophoresis.

[27]  Jin Cui,et al.  Multi-bearing remaining useful life collaborative prediction: A deep learning approach , 2017 .

[28]  Angelo Carfì,et al.  Persistent Symptoms in Patients After Acute COVID-19. , 2020, JAMA.

[29]  S. Cessie,et al.  Ridge Estimators in Logistic Regression , 1992 .

[30]  Thomas Niesler,et al.  COVID-19 cough classification using machine learning and global smartphone recordings , 2021, Computers in Biology and Medicine.

[31]  J. Korpáš,et al.  Analysis of the cough sound: an overview. , 1996, Pulmonary pharmacology.

[32]  Madhurananda Pahar,et al.  Coding and Decoding Speech using a Biologically Inspired Coding System , 2020, 2020 IEEE Symposium Series on Computational Intelligence (SSCI).

[33]  Juliana A. Knocikova,et al.  Wavelet analysis of voluntary cough sound in patients with respiratory diseases. , 2008, Journal of physiology and pharmacology : an official journal of the Polish Physiological Society.

[34]  Bartosz Krawczyk,et al.  Learning from imbalanced data: open challenges and future directions , 2016, Progress in Artificial Intelligence.

[35]  J. Tukey,et al.  Modern techniques of power spectrum estimation , 1967, IEEE Transactions on Audio and Electroacoustics.

[36]  Taghi M. Khoshgoftaar,et al.  Experimental perspectives on learning from imbalanced data , 2007, ICML '07.

[37]  Udantha R. Abeyratne,et al.  Cough sound analysis for diagnosing croup in pediatric patients using biologically inspired features , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[38]  Buket D. Barkana,et al.  Voiced/Unvoiced Decision for Speech Signals Based on Zero-Crossing Rate and Energy , 2008, SCSS.

[39]  Thomas R. Niesler,et al.  A Comparative Study of Features for Acoustic Cough Detection Using Deep Architectures* , 2019, 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[40]  Brian Tracey,et al.  Cough detection algorithm for monitoring patient recovery from pulmonary tuberculosis , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[41]  A Bush,et al.  A new device for ambulatory cough recording , 1994, Pediatric pulmonology.

[42]  SYSTEMIS AND METHODS FOR MONITORING COUGH , 2017 .

[43]  Tracey J. Mehigan Harnessing accelerometer technology for inclusive mobile learning , 2009, Mobile HCI.

[44]  Renard Xaviero Adhi Pramono,et al.  A Cough-Based Algorithm for Automatic Diagnosis of Pertussis , 2016, PloS one.

[45]  Eyal de Lara,et al.  Coughwatch: Real-World Cough Detection using Smartwatches , 2021, ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[46]  Ian M Paul,et al.  Evaluation of a new self-contained, ambulatory, objective cough monitor , 2006, Cough.

[47]  Erik Marchi,et al.  Non-linear prediction with LSTM recurrent neural networks for acoustic novelty detection , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[48]  Martin Paegelow,et al.  Geomatic Approaches for Modeling Land Change Scenarios , 2018 .

[49]  Mihir Narayan Mohanty,et al.  Design of MLP Based Model for Analysis of Patient Suffering from Influenza , 2016 .

[50]  Thomas Niesler,et al.  Deep Neural Network Based Cough Detection Using Bed-Mounted Accelerometer Measurements , 2021, ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[51]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[52]  P D van Helden,et al.  Detection of tuberculosis by automatic cough sound analysis , 2018, Physiological measurement.

[53]  Kofi Odame,et al.  Deep Neural Networks for Identifying Cough Sounds , 2016, IEEE Transactions on Biomedical Circuits and Systems.

[54]  Zheng Wang,et al.  Cough detection using deep neural networks , 2014, 2014 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[55]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[56]  John Adcock,et al.  Audio privacy: reducing speech intelligibility while preserving environmental sounds , 2008, ACM Multimedia.

[57]  Yu-Chih Tung,et al.  Exploiting Sound Masking for Audio Privacy in Smartphones , 2019, AsiaCCS.

[58]  Thomas Niesler,et al.  Automatic cough classification for tuberculosis screening in a real-world environment , 2021, Physiological measurement.

[59]  Frank Knoefel,et al.  Feature extraction for the differentiation of dry and wet cough sounds , 2011, 2011 IEEE International Symposium on Medical Measurements and Applications.

[60]  Mahmood Al-khassaweneh,et al.  A signal processing approach for the diagnosis of asthma from cough sounds , 2013, Journal of medical engineering & technology.

[61]  Hind Taud,et al.  Multilayer Perceptron (MLP) , 2018 .

[62]  Kofi Odame,et al.  DeepCough: A deep convolutional neural network in a wearable cough detection system , 2015, 2015 IEEE Biomedical Circuits and Systems Conference (BioCAS).

[63]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[64]  Vincent Rouillard,et al.  On the Use of Machine Learning to Detect Shocks in Road Vehicle Vibration Signals , 2017 .

[65]  S. Subburaj,et al.  Application and validation of a computerized cough acquisition system for objective monitoring of acute cough: a meta-analysis. , 2001, Chest.

[66]  Xingqun Qi,et al.  Comparison of Support Vector Machine and Softmax Classifiers in Computer Vision , 2017, 2017 Second International Conference on Mechanical, Control and Computer Engineering (ICMCCE).

[67]  Gaël Richard,et al.  Temporal Integration for Audio Classification With Application to Musical Instrument Classification , 2009, IEEE Transactions on Audio, Speech, and Language Processing.

[68]  Brian Subirana,et al.  COVID-19 Artificial Intelligence Diagnosis Using Only Cough Recordings , 2020, IEEE Open Journal of Engineering in Medicine and Biology.

[69]  Vikrant Bhateja,et al.  Pre-Processing and Classification of Cough Sounds in Noisy Environment using SVM , 2019, 2019 4th International Conference on Information Systems and Computer Networks (ISCON).

[70]  Fernando Nogueira,et al.  Imbalanced-learn: A Python Toolbox to Tackle the Curse of Imbalanced Datasets in Machine Learning , 2016, J. Mach. Learn. Res..

[71]  Sophia Ananiadou,et al.  Stochastic Gradient Descent Training for L1-regularized Log-linear Models with Cumulative Penalty , 2009, ACL.

[72]  Ping Zhou,et al.  Machine Learning for Supporting Diagnosis of Amyotrophic Lateral Sclerosis Using Surface Electromyogram , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[73]  Nobutaka Ono,et al.  ACOUSTIC SCENE CLASSIFICATION USING DEEP NEURAL NETWORK AND FRAME-CONCATENATED ACOUSTIC FEATURE , 2016 .

[74]  Simon Iwnicki,et al.  Application of power spectrum, cepstrum, higher order spectrum and neural network analyses for induction motor fault diagnosis , 2013 .

[75]  Robert L. Lux,et al.  The Application of Root Mean Square Electrocardiography (RMS ECG) for the Detection of Acquired and Congenital Long QT Syndrome , 2014, PloS one.