Attention Autoencoder for Generative Latent Representational Learning in Anomaly Detection

Today, accurate and automated abnormality diagnosis and identification have become of paramount importance as they are involved in many critical and life-saving scenarios. To accomplish such frontiers, we propose three artificial intelligence models through the application of deep learning algorithms to analyze and detect anomalies in human heartbeat signals. The three proposed models include an attention autoencoder that maps input data to a lower-dimensional latent representation with maximum feature retention, and a reconstruction decoder with minimum remodeling loss. The autoencoder has an embedded attention module at the bottleneck to learn the salient activations of the encoded distribution. Additionally, a variational autoencoder (VAE) and a long short-term memory (LSTM) network is designed to learn the Gaussian distribution of the generative reconstruction and time-series sequential data analysis. The three proposed models displayed outstanding ability to detect anomalies on the evaluated five thousand electrocardiogram (ECG5000) signals with 99% accuracy and 99.3% precision score in detecting healthy heartbeats from patients with severe congestive heart failure.

[1]  Andrew Y. Ng,et al.  Cardiologist-Level Arrhythmia Detection with Convolutional Neural Networks , 2017, ArXiv.

[2]  Q. M. Jonathan Wu,et al.  MAMA Net: Multi-Scale Attention Memory Autoencoder Network for Anomaly Detection , 2020, IEEE Transactions on Medical Imaging.

[3]  Houshang Darabi,et al.  LSTM Fully Convolutional Networks for Time Series Classification , 2017, IEEE Access.

[4]  Waqas Rasheed,et al.  Anomaly Detection of Moderate Traumatic Brain Injury Using Auto-Regularized Multi-Instance One-Class SVM , 2020, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[5]  Hyperspectral Image Classification—Traditional to Deep Models: A Survey for Future Prospects , 2021, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[6]  M. Rezghi,et al.  End-to-end CNN + LSTM deep learning approach for bearing fault diagnosis , 2019, Applied Intelligence.

[7]  Lovekesh Vig,et al.  TimeNet: Pre-trained deep recurrent neural network for time series classification , 2017, ESANN.

[8]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[9]  Eatedal Alabdulkreem,et al.  CaptionNet: Automatic End-to-End Siamese Difference Captioning Model With Attention , 2019, IEEE Access.

[10]  Christopher D. Manning,et al.  Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks , 2015, ACL.

[11]  Houshang Darabi,et al.  Multivariate LSTM-FCNs for Time Series Classification , 2018, Neural Networks.

[12]  Chuang Sun,et al.  Deep Coupling Autoencoder for Fault Diagnosis With Multimodal Sensory Data , 2018, IEEE Transactions on Industrial Informatics.

[13]  Florence Forbes,et al.  Fully Automatic Lesion Localization and Characterization: Application to Brain Tumors Using Multiparametric Quantitative MRI Data , 2018, IEEE Transactions on Medical Imaging.

[14]  Gongjian Wen,et al.  Video Anomaly Detection and Localization via Gaussian Mixture Fully Convolutional Variational Autoencoder , 2018, Comput. Vis. Image Underst..

[15]  Yaguo Lei,et al.  Applications of machine learning to machine fault diagnosis: A review and roadmap , 2020 .

[16]  Raghavendra Chalapathy University of Sydney,et al.  Deep Learning for Anomaly Detection: A Survey , 2019, ArXiv.

[17]  Jinghui Qiao,et al.  Abnormal Condition Detection Integrated with Kullback Leibler Divergence and Relative Importance Function for Cement Raw Meal Calcination Process , 2020, 2020 2nd International Conference on Industrial Artificial Intelligence (IAI).

[18]  Xin Zhou,et al.  Deep neural networks: A promising tool for fault characteristic mining and intelligent diagnosis of rotating machinery with massive data , 2016 .

[19]  Daren Yu,et al.  Long Short-Term Memory Network-Based Normal Pattern Group for Fault Detection of Three-Shaft Marine Gas Turbine , 2020, Energies.

[20]  Na Qin,et al.  Convolutional Recurrent Neural Network for Fault Diagnosis of High-Speed Train Bogie , 2018, Complex.

[21]  Zhao Ke,et al.  A novel tracking deep wavelet auto-encoder method for intelligent fault diagnosis of electric locomotive bearings , 2018, Mechanical Systems and Signal Processing.

[22]  Chen Lu,et al.  Fault diagnosis of rotary machinery components using a stacked denoising autoencoder-based health state identification , 2017, Signal Process..

[23]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[24]  Muhammad Umar Aftab,et al.  Attentively Conditioned Generative Adversarial Network for Semantic Segmentation , 2020, IEEE Access.

[25]  Margarida Silveira,et al.  Learning Representations from Healthcare Time Series Data for Unsupervised Anomaly Detection , 2019, 2019 IEEE International Conference on Big Data and Smart Computing (BigComp).

[26]  Manuel Mazzara,et al.  Regularized CNN Feature Hierarchy for Hyperspectral Image Classification , 2021, Remote. Sens..

[27]  Abien Fred Agarap Deep Learning using Rectified Linear Units (ReLU) , 2018, ArXiv.

[28]  Xian-Bo Wang,et al.  Representational Learning for Fault Diagnosis of Wind Turbine Equipment: A Multi-Layered Extreme Learning Machines Approach , 2016 .

[29]  Jan Kautz,et al.  NVAE: A Deep Hierarchical Variational Autoencoder , 2020, NeurIPS.

[30]  Weijie Wang,et al.  Analysis of the Mean Absolute Error (MAE) and the Root Mean Square Error (RMSE) in Assessing Rounding Model , 2018 .

[31]  Hugo Gamboa,et al.  Robust Anomaly Detection in Time Series through Variational AutoEncoders and a Local Similarity Score , 2021, BIOSIGNALS.

[32]  Eamonn J. Keogh,et al.  A general framework for never-ending learning from time series streams , 2015, Data Mining and Knowledge Discovery.

[33]  Ronen Feldman,et al.  The Data Mining and Knowledge Discovery Handbook , 2005 .

[34]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[35]  Zhiguang Qin,et al.  Fully Convolutional CaptionNet: Siamese Difference Captioning Attention Model , 2019, IEEE Access.

[36]  T. Brotherton,et al.  Anomaly detection for advanced military aircraft using neural networks , 2001, 2001 IEEE Aerospace Conference Proceedings (Cat. No.01TH8542).

[37]  Jeffrey M. Hausdorff,et al.  Physionet: Components of a New Research Resource for Complex Physiologic Signals". Circu-lation Vol , 2000 .

[38]  Terence Soule,et al.  A Review of Local Outlier Factor Algorithms for Outlier Detection in Big Data Streams , 2020, Big Data Cogn. Comput..

[39]  Guan Gui,et al.  Semi-Supervised Machine Learning Aided Anomaly Detection Method in Cellular Networks , 2020, IEEE Transactions on Vehicular Technology.

[40]  Chin-Wei Tien,et al.  Using Autoencoders for Anomaly Detection and Transfer Learning in IoT , 2021, Comput..