Deep Learning-Based Model Architecture for Time-Frequency Images Analysis

Time-frequency analysis is an initial step in the design of invariant representations for any type of time series signals. Time-frequency analysis has been studied and developed widely for decades, but accurate analysis using deep learning neural networks has only been presented in the last few years. In this paper, a comprehensive survey of deep learning neural network architectures for time-frequency analysis is presented and compares the networks with previous approaches to time-frequency analysis based on feature extraction and other machine learning algorithms. The results highlight the improvements achieved by deep learning networks, critically review the application of deep learning for time-frequency analysis and provide a holistic overview of current works in the literature. Finally, this work facilitates discussions regarding research opportunities with deep learning algorithms in future researches.

[1]  Lars Kai Hansen,et al.  Deep convolutional neural networks for interpretable analysis of EEG sleep stage scoring , 2017, 2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP).

[2]  Dan Stowell,et al.  Detection and Classification of Acoustic Scenes and Events , 2015, IEEE Transactions on Multimedia.

[3]  Haya Alaskar.  Dynamic self-organised neural network inspired by the immune algorithm for financial time series prediction and medical data classification , 2014 .

[4]  Giorgio Biagetti,et al.  Surface EMG Fatigue Analysis by Means of Homomorphic Deconvolution , 2014, mBiDA.

[5]  Yike Guo,et al.  Automatic Sleep Stage Scoring with Single-Channel EEG Using Convolutional Neural Networks , 2016, ArXiv.

[6]  Christina Gloeckner.  Foundations Of Time Frequency Analysis , 2016 .

[7]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups , 2012, IEEE Signal Processing Magazine.

[8]  Lixiang Duan,et al.  A multi-scale convolution neural network for featureless fault diagnosis , 2016, 2016 International Symposium on Flexible Automation (ISFA).

[9]  Oluwarotimi Williams Samuel,et al.  Towards Real-Time Detection of Gait Events on Different Terrains Using Time-Frequency Analysis and Peak Heuristics Algorithm , 2016, Sensors.

[10]  Filip De Turck,et al.  A novel time series analysis approach for prediction of dialysis in critically ill patients using echo-state networks , 2010, BMC Medical Informatics Decis. Mak..

[11]  Sridhar Krishnan,et al.  Deep Learning of EEG Time-Frequency Representations for Identifying Eye States , 2018, Adv. Data Sci. Adapt. Anal..

[12]  Amy Loutfi,et al.  Sleep Stage Classification Using Unsupervised Feature Learning , 2012, Adv. Artif. Neural Syst..

[13]  R. Palaniappan,et al.  Classification of biological signals using linear and nonlinear features , 2010, Physiological measurement.

[14]  Abir Jaafar Hussain,et al.  Data Mining to Support the Discrimination of Amyotrophic Lateral Sclerosis Diseases Based on Gait Analysis , 2018, ICIC.

[15]  Maarten De Vos,et al.  Visualising convolutional neural network decisions in automated sleep scoring , 2018, AIH@IJCAI.

[16]  Giulio Ruffini,et al.  Deep Learning With EEG Spectrograms in Rapid Eye Movement Behavior Disorder , 2018, bioRxiv.

[17]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Steven Verstockt,et al.  Convolutional Neural Network Based Fault Detection for Rotating Machinery , 2016 .

[19]  Robert Keight,et al.  Predicting Freezing of Gait in Parkinsons Disease Patients Using Machine Learning , 2018, 2018 IEEE Congress on Evolutionary Computation (CEC).

[20]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[21]  Lanlan Yu,et al.  Study of Feature Classification Methods in BCI Based on Neural Networks , 2005, 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference.

[22]  M. Elgendi,et al.  Photoplethysmography and Deep Learning: Enhancing Hypertension Risk Stratification , 2018, Biosensors.

[23]  Jichao Zhao,et al.  Robust ECG signal classification for detection of atrial fibrillation using a novel neural network , 2017, 2017 Computing in Cardiology (CinC).

[24]  Shalin Savalia,et al.  Cardiac Arrhythmia Classification by Multi-Layer Perceptron and Convolution Neural Networks , 2018, Bioengineering.

[25]  Norden E. Huang,et al.  INTRODUCTION TO THE HILBERT–HUANG TRANSFORM AND ITS RELATED MATHEMATICAL PROBLEMS , 2005 .

[26]  Masakiyo Fujimoto,et al.  Exploiting spectro-temporal locality in deep learning based acoustic event detection , 2015, EURASIP J. Audio Speech Music. Process..

[27]  Mohammad Modarres,et al.  Deep Learning Enabled Fault Diagnosis Using Time-Frequency Image Analysis of Rolling Element Bearings , 2017 .

[28]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[29]  Omid Dehzangi,et al.  IMU-Based Gait Recognition Using Convolutional Neural Networks and Multi-Sensor Fusion , 2017, Sensors.

[30]  Björn W. Schuller,et al.  Learning Image-based Representations for Heart Sound Classification , 2018, DH.

[31]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[32]  Muhammad Huzaifah,et al.  Comparison of Time-Frequency Representations for Environmental Sound Classification using Convolutional Neural Networks , 2017, ArXiv.

[33]  George Saon,et al.  Analyzing convolutional neural networks for speech activity detection in mismatched acoustic conditions , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[34]  Yinghong Peng,et al.  EMG‐Based Estimation of Limb Movement Using Deep Learning With Recurrent Convolutional Neural Networks , 2018, Artificial organs.

[35]  Moncef Gabbouj,et al.  Real-Time Patient-Specific ECG Classification by 1-D Convolutional Neural Networks , 2016, IEEE Transactions on Biomedical Engineering.

[36]  Suet-Peng Yong,et al.  An Evaluation of Convolutional Neural Nets for Medical Image Anatomy Classification , 2016 .

[37]  Jianqing Li,et al.  Patient-Specific Deep Architectural Model for ECG Classification , 2017, Journal of healthcare engineering.

[38]  Audun Eltvik,et al.  Deep Learning for the Classification of EEG Time-Frequency Representations , 2018 .

[39]  Charles Yetman,et al.  Convolutional Neural Net and Bearing Fault Analysis , 2016 .

[40]  U. Rajendra Acharya,et al.  Automated characterization of arrhythmias using nonlinear features from tachycardia ECG beats , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[41]  U. Rajendra Acharya,et al.  Deep convolutional neural network for the automated detection and diagnosis of seizure using EEG signals , 2017, Comput. Biol. Medicine.

[42]  Hissam Tawfik,et al.  Feature Analysis of Uterine Electrohystography Signal Using Dynamic Self-organised Multilayer Network Inspired by the Immune Algorithm , 2014, ICIC.

[43]  U. Rajendra Acharya,et al.  Application of deep convolutional neural network for automated detection of myocardial infarction using ECG signals , 2017, Inf. Sci..

[44]  Stephen J. Roberts,et al.  Mosquito Detection with Neural Networks: The Buzz of Deep Learning , 2017, ArXiv.

[45]  Longhao Yuan,et al.  Patients' EEG Data Analysis via Spectrogram Image with a Convolution Neural Network , 2017, KES-IDT.

[46]  Wei Gao,et al.  A Novel Fault Diagnosis Method for Rotating Machinery Based on a Convolutional Neural Network , 2018, Sensors.

[47]  Shao-Hu Peng,et al.  Acoustic Scene Classification Using Deep Convolutional Neural Network and Multiple Spectrograms Fusion , 2017, DCASE.

[48]  Bao-Liang Lu,et al.  Investigating Critical Frequency Bands and Channels for EEG-Based Emotion Recognition with Deep Neural Networks , 2015, IEEE Transactions on Autonomous Mental Development.

[49]  Y. S. Rao,et al.  SVM based machine learning approach to identify Parkinson's disease using gait analysis , 2016, 2016 International Conference on Inventive Computation Technologies (ICICT).

[50]  Vili Podgorelec,et al.  Automatic Classification of Motor Impairment Neural Disorders from EEG Signals Using Deep Convolutional Neural Networks , 2018, Elektronika ir Elektrotechnika.

[51]  Maarten De Vos,et al.  Comparing feature-based classifiers and convolutional neural networks to detect arrhythmia from short segments of ECG , 2017, 2017 Computing in Cardiology (CinC).

[52]  Dingguo Zhang,et al.  A discriminant bispectrum feature for surface electromyogram signal classification. , 2010, Medical engineering & physics.

[53]  Hongmei Liu,et al.  Rolling Bearing Fault Diagnosis Based on STFT-Deep Learning and Sound Signals , 2016 .

[54]  Gee-Sern Hsu,et al.  Deep learning with time-frequency representation for pulse estimation from facial videos , 2017, 2017 IEEE International Joint Conference on Biometrics (IJCB).