Deep learning based classification of unsegmented phonocardiogram spectrograms leveraging transfer learning

Objective. Cardiovascular diseases (CVDs) are a main cause of deaths all over the world. This research focuses on computer-aided analysis of phonocardiogram (PCG) signals based on deep learning that can enable improved and timely detection of heart abnormalities. The two widely used publicly available PCG datasets are from the PhysioNet/CinC (2016) and PASCAL (2011) challenges. The datasets are significantly different in terms of the tools used for data acquisition, clinical protocols, digital storages and signal qualities, making it challenging to process and analyze. Approach. In this work, we have used short-time Fourier transform-based spectrograms to learn the representative patterns of the normal and abnormal PCG signals. Spectrograms generated from both the datasets are utilized to perform four different studies: (i) train, validate and test different variants of convolutional neural network (CNN) models with PhysioNet dataset, (ii) train, validate and test the best performing CNN structure on the PASCAL dataset, as well as (iii) on the combined PhysioNet-PASCAL dataset and (iv) finally, the transfer learning technique is employed to train the best performing pre-trained network from the first study with PASCAL dataset. Main results. The first study achieves an accuracy, sensitivity, specificity, precision and F1 scores of 95.75%, 96.3%, 94.1%, 97.52%, and 96.93%, respectively, while the second study shows accuracy, sensitivity, specificity, precision and F1 scores of 75.25%, 74.2%, 76.4%, 76.73%, and 75.42%, respectively. The third study shows accuracy, sensitivity, specificity, precision and F1 scores of 92.7%, 94.98%, 89.95%, 95.3% and 94.6%, respectively. Finally, the fourth study shows a precision of 96.98% on the noisy PASCAL dataset with transfer learning approach. Significance. The proposed approach employs a less complex and relatively light custom CNN model that outperforms most of the recent competing studies by achieving comparatively high classification accuracy and precision, making it suitable for screening CVDs using PCG signals.

[1]  R. Maas,et al.  A summary of the REVERB challenge: state-of-the-art and remaining challenges in reverberant speech processing research , 2016, EURASIP Journal on Advances in Signal Processing.

[2]  Snekhalatha Umapathy,et al.  Automated heart sound classification system from unsegmented phonocardiogram (PCG) using deep neural network , 2020, Physical and Engineering Sciences in Medicine.

[3]  Jürgen Schmidhuber,et al.  Multi-column deep neural networks for image classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  James Bailey,et al.  Understanding Adversarial Attacks on Deep Learning Based Medical Image Analysis Systems , 2019, Pattern Recognit..

[5]  Gabriel Jimenez-Moreno,et al.  Deep Neural Networks for the Recognition and Classification of Heart Murmurs Using Neuromorphic Auditory Sensors , 2017, IEEE Transactions on Biomedical Circuits and Systems.

[6]  Wenjie Zhang,et al.  Towards heart sound classification without segmentation using convolutional neural network , 2017, 2017 Computing in Cardiology (CinC).

[7]  Qiao Li,et al.  An open access database for the evaluation of heart sound algorithms , 2016, Physiological measurement.

[8]  Shengli Xie,et al.  Heart sound classification using SNMFNet classifier. , 2019, Physiological measurement.

[9]  Chunmei Qing,et al.  Classification of Heart Sounds Based on Convolutional Neural Network , 2017, ICIMCS.

[10]  Abdelghani Djebbari,et al.  Short-time Fourier transform analysis of the phonocardiogram signal , 2000, ICECS 2000. 7th IEEE International Conference on Electronics, Circuits and Systems (Cat. No.00EX445).

[11]  Yannis Stylianou,et al.  A study of time-frequency features for CNN-based automatic heart sound classification for pathology detection , 2018, Comput. Biol. Medicine.

[12]  B. Prendergast,et al.  Cardiac auscultation poorly predicts the presence of valvular heart disease in asymptomatic primary care patients , 2018, Heart.

[13]  Muhammad Imran Razzak,et al.  Deep Learning for Medical Image Processing: Overview, Challenges and Future , 2017, ArXiv.

[14]  Eliasz Kantoch,et al.  A Deep Learning Approach for Valve Defect Recognition in Heart Acoustic Signal , 2017, ISAT.

[15]  Rocío Alba-Flores,et al.  Recognition and classification of cardiac murmurs using ANN and segmentation , 2012, CONIELECOMP 2012, 22nd International Conference on Electrical Communications and Computers.

[16]  R. Sepponen,et al.  Phono-spectrographic analysis of heart murmur in children , 2007, BMC pediatrics.

[17]  Paul R. White,et al.  Classification of heart sounds using time-frequency method and artificial neural networks , 2000, Proceedings of the 22nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (Cat. No.00CH37143).

[18]  Richard K. G. Do,et al.  Convolutional neural networks: an overview and application in radiology , 2018, Insights into Imaging.

[19]  Abir Hussain,et al.  The Implementation of Pretrained AlexNet on PCG Classification , 2019, ICIC.

[20]  Venkatesan Rajinikanth,et al.  Deep Learning for Medical Image Processing , 2020 .

[21]  Imran Siddiqi,et al.  Localization and classification of heartbeats using robust adaptive algorithm , 2019, Biomed. Signal Process. Control..

[22]  Omer Deperlioglu,et al.  Classification of Segmented Phonocardiograms by Convolutional Neural Networks , 2019 .

[23]  A K Mittra,et al.  Time-frequency analysis of foetal heart sound signal for the prediction of prenatal anomalies , 2009, Journal of medical engineering & technology.

[24]  Rayaz A. Malik,et al.  Cardiac auscultation: an essential clinical skill in decline , 2010 .

[25]  Yifan Gong,et al.  Cross-language knowledge transfer using multilingual deep neural network with shared hidden layers , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[26]  Yu Tsao,et al.  S1 and S2 Heart Sound Recognition Using Deep Neural Networks , 2017, IEEE Trans. Biomed. Eng..

[27]  Solomon Teferra Abate,et al.  DNN-Based Speech Recognition for Globalphone Languages , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[28]  Qiao Li,et al.  Classification of normal/abnormal heart sound recordings: The PhysioNet/Computing in Cardiology Challenge 2016 , 2016, 2016 Computing in Cardiology Conference (CinC).

[29]  J. Gray,et al.  Helping family physicians improve their cardiac auscultation skills with an interactive CD‐ROM , 2002, The Journal of continuing education in the health professions.

[30]  Soma Bandyopadhyay,et al.  Murmur Detection Using Parallel Recurrent & Convolutional Neural Networks , 2018, ArXiv.

[31]  Franz Pernkopf,et al.  DNN-based speech mask estimation for eigenvector beamforming , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[32]  Miguel Sotaquirá,et al.  Phonocardiogram classification using deep neural networks and weighted probability comparisons , 2018, Journal of medical engineering & technology.

[33]  I Md. Dendi Maysanjaya,et al.  Abnormal Heart Rhythm Detection Based on Spectrogram of Heart Sound using Convolutional Neural Network , 2018, 2018 6th International Conference on Cyber and IT Service Management (CITSM).

[34]  Bryan R. Conroy,et al.  Ensemble of feature-based and deep learning-based classifiers for detection of abnormal heart sounds , 2016, 2016 Computing in Cardiology Conference (CinC).

[35]  Marta R. Costa-jussà,et al.  Introduction to the special issue on deep learning approaches for machine translation , 2017, Comput. Speech Lang..

[36]  Ahmed Hammouch,et al.  Phonocardiogram signals processing approach for PASCAL Classifying Heart Sounds Challenge , 2018, Signal Image Video Process..

[37]  Siddique Latif,et al.  Mobile Technologies for Managing Non-Communicable Diseases in Developing Countries , 2018 .

[38]  Mohd Zubir Suboh,et al.  Classification of heart sound based on s-transform and neural network , 2010, 10th International Conference on Information Science, Signal Processing and their Applications (ISSPA 2010).

[39]  Stephen Marshall,et al.  Activation Functions: Comparison of trends in Practice and Research for Deep Learning , 2018, ArXiv.

[40]  Mamun Bin Ibne Reaz,et al.  Real-Time Smart-Digital Stethoscope System for Heart Diseases Monitoring , 2019, Sensors.

[41]  Yan Xu,et al.  Deep learning of feature representation with multiple instance learning for medical image analysis , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[42]  Qixing Zhang,et al.  Wildland Forest Fire Smoke Detection Based on Faster R-CNN using Synthetic Smoke Images , 2018 .

[43]  Yannis Stylianou,et al.  Automatic classification of systolic heart murmurs , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[44]  Varun Bajaj,et al.  Towards the classification of heart sounds based on convolutional deep neural network , 2019, Health Information Science and Systems.

[45]  N. D. Nanayakkara,et al.  Automated diagnosis of cardiac abnormalities using heart sounds , 2013, 2013 IEEE Point-of-Care Healthcare Technologies (PHT).

[46]  Rajib Rana,et al.  Phonocardiographic Sensing Using Deep Learning for Abnormal Heartbeat Detection , 2018, IEEE Sensors Journal.

[47]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups , 2012, IEEE Signal Processing Magazine.

[48]  Kumar Sricharan,et al.  Recognizing Abnormal Heart Sounds Using Deep Learning , 2017, KHD@IJCAI.

[49]  A.Z. Sha'ameri,et al.  Time-frequency analysis of heart sounds and murmurs , 2003, Fourth International Conference on Information, Communications and Signal Processing, 2003 and the Fourth Pacific Rim Conference on Multimedia. Proceedings of the 2003 Joint.

[50]  Anam Abid,et al.  Localization of Phonocardiogram Signals Using Multi-level Threshold and Support Vector Machine , 2019, 2019 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT).

[51]  Chee-Ming Ting,et al.  Short-segment Heart Sound Classification Using an Ensemble of Deep Convolutional Neural Networks , 2018, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[52]  Wenjie Zhang,et al.  Heart sound classification based on scaled spectrogram and tensor decomposition , 2017, Expert Syst. Appl..

[53]  Hayong Shin,et al.  Classification of heart sound recordings using convolution neural network , 2016, 2016 Computing in Cardiology Conference (CinC).

[54]  Xiaodai Dong,et al.  Life-Threatening Ventricular Arrhythmia Detection With Personalized Features , 2017, IEEE Access.

[55]  Amith Khandakar,et al.  Machine Learning Based Photovoltaics (PV) Power Prediction Using Different Environmental Parameters of Qatar , 2019, Energies.

[56]  Swanirbhar Majumder,et al.  CLASSIFICATION OF UNSEGMENTED HEART SOUND RECORDING USING KNN CLASSIFIER , 2019, Journal of Mechanics in Medicine and Biology.

[57]  Krzysztof Wołk,et al.  Early and Remote Detection of Possible Heartbeat Problems With Convolutional Neural Networks and Multipart Interactive Training , 2019, IEEE Access.

[58]  Anirban Mukherjee,et al.  Characterization of $S_1$ and $S_2$ Heart Sounds Using Stacked Autoencoder and Convolutional Neural Network , 2019, IEEE Transactions on Instrumentation and Measurement.

[59]  Jiajun Wu,et al.  Deep multiple instance learning for image classification and auto-annotation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[60]  Aggelos K. Katsaggelos,et al.  Heart sound anomaly and quality detection using ensemble of neural networks without segmentation , 2016, 2016 Computing in Cardiology Conference (CinC).

[61]  Jianxi Yang,et al.  ResHNet: Spectrograms Based Efficient Heart Sounds Classification Using Stacked Residual Networks , 2019, 2019 IEEE EMBS International Conference on Biomedical & Health Informatics (BHI).

[62]  Amith Khandakar,et al.  Wearable Real-Time Heart Attack Detection and Warning System to Reduce Road Accidents , 2019, Sensors.

[63]  K. P. Soman,et al.  Deep models for phonocardiography (PCG) classification , 2017, 2017 International Conference on Intelligent Communication and Computational Techniques (ICCT).

[64]  Shi-Wen Deng,et al.  Towards heart sound classification without segmentation via autocorrelation feature and diffusion maps , 2016, Future Gener. Comput. Syst..

[65]  Li Ke,et al.  Classification of Heart Sounds Based on the Wavelet Fractal and Twin Support Vector Machine , 2019, Entropy.

[66]  Muhammad Salman Khan,et al.  Automatic heart sound classification from segmented/unsegmented phonocardiogram signals using time and frequency features , 2020, Physiological measurement.

[67]  Xindong Wu,et al.  Object Detection With Deep Learning: A Review , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[68]  Gyu Sang Choi,et al.  Heartbeat Sound Signal Classification Using Deep Learning , 2019, Sensors.

[69]  Manal Abdel Wahed,et al.  Classification of heart sounds using fractional fourier transform based mel-frequency spectral coefficients and traditional classifiers , 2020, Biomed. Signal Process. Control..