Anomaly Detection for Sensor Signals Utilizing Deep Learning Autoencoder-Based Neural Networks

Anomaly detection is a significant task in sensors’ signal processing since interpreting an abnormal signal can lead to making a high-risk decision in terms of sensors’ applications. Deep learning algorithms are effective tools for anomaly detection due to their capability to address imbalanced datasets. In this study, we took a semi-supervised learning approach, utilizing normal data for training the deep learning neural networks, in order to address the diverse and unknown features of anomalies. We developed autoencoder-based prediction models to automatically detect anomalous data recorded by three electrochemical aptasensors, with variations in the signals’ lengths for particular concentrations, analytes, and bioreceptors. Prediction models employed autoencoder networks and the kernel density estimation (KDE) method for finding the threshold to detect anomalies. Moreover, the autoencoder networks were vanilla, unidirectional long short-term memory (ULSTM), and bidirectional LSTM (BLSTM) autoencoders for the training stage of the prediction models. However, the decision-making was based on the result of these three networks and the integration of vanilla and LSTM networks’ results. The accuracy as a performance metric of anomaly prediction models showed that the performance of vanilla and integrated models were comparable, while the LSTM-based autoencoder models showed the least accuracy. Considering the integrated model of ULSTM and vanilla autoencoder, the accuracy for the dataset with the lengthier signals was approximately 80%, while it was 65% and 40% for the other datasets. The lowest accuracy belonged to the dataset with the least normal data in its dataset. These results demonstrate that the proposed vanilla and integrated models can automatically detect abnormal data when there is sufficient normal data for training the models.

[1]  Igor Kotenko,et al.  Detection of Cyberattacks and Anomalies in Cyber-Physical Systems: Approaches, Data Sources, Evaluation , 2023, Algorithms.

[2]  Hafiz Tayyab Rauf,et al.  Bone Anomaly Detection by Extracting Regions of Interest and Convolutional Neural Networks , 2023, Applied System Innovation.

[3]  Won-Ju Lee,et al.  Anomaly Detection Based on Time Series Data of Hydraulic Accumulator , 2022, Sensors.

[4]  Branka Stojanović,et al.  Robust Financial Fraud Alerting System Based in the Cloud Environment , 2022, Sensors.

[5]  Gen Li,et al.  Deep learning for anomaly detection in multivariate time series: Approaches, applications, and challenges , 2022, Inf. Fusion.

[6]  C. Unsworth,et al.  Predicting Analyte Concentrations from Electrochemical Aptasensor Signals Using LSTM Recurrent Networks , 2022, Bioengineering.

[7]  Naveed Iqbal DeepSeg: Deep Segmental Denoising Neural Network for Seismic Data , 2022, IEEE Transactions on Neural Networks and Learning Systems.

[8]  X. Marimon,et al.  Detection of Abnormal Cardiac Response Patterns in Cardiac Tissue Using Deep Learning , 2022, Mathematics.

[9]  Youchen Fan,et al.  ResNet-AE for Radar Signal Anomaly Detection , 2022, Sensors.

[10]  M. Schram,et al.  Time series anomaly detection in power electronics signals with recurrent and ConvLSTM autoencoders , 2022, Digit. Signal Process..

[11]  Suyang Zhou,et al.  An Anomaly Detection Method of Time Series Data for Cyber-Physical Integrated Energy System Based on Time-Frequency Feature Prediction , 2022, Energies.

[12]  W. Yeo,et al.  Automatic and Accurate Sleep Stage Classification via a Convolutional Deep Neural Network and Nanomembrane Electrodes , 2022, Biosensors.

[13]  Weifeng Zhang,et al.  Arrhythmia classification of LSTM autoencoder based on time series anomaly detection , 2022, Biomed. Signal Process. Control..

[14]  Nikolaos G. Paterakis,et al.  Data augmentation for time series regression: Applying transformations, autoencoders and adversarial networks to electricity price forecasting , 2021, Applied Energy.

[15]  Nicholas R. Jennings,et al.  Unsupervised anomaly detection with LSTM autoencoders using statistical data-filtering , 2021, Appl. Soft Comput..

[16]  N. Plank,et al.  Comparison of Duplex and Quadruplex Folding Structure Adenosine Aptamers for Carbon Nanotube Field Effect Transistor Aptasensors , 2021, Nanomaterials.

[17]  En Zhu,et al.  Improved autoencoder for unsupervised anomaly detection , 2021, Int. J. Intell. Syst..

[18]  L. Girish,et al.  Anomaly detection in cloud environment using artificial intelligence techniques , 2021, Computing.

[19]  Souran Manoochehri,et al.  Online Convolutional Neural Network-based anomaly detection and quality control for Fused Filament Fabrication process , 2021 .

[20]  Jinbo Li,et al.  Clustering-based anomaly detection in multivariate time series data , 2021, Appl. Soft Comput..

[21]  Naveed Iqbal,et al.  Automated Event Detection and Denoising Method for Passive Seismic Data Using Residual Deep Convolutional Neural Networks , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[22]  Huu Du Nguyen,et al.  Forecasting and Anomaly Detection approaches using LSTM and LSTM Autoencoder techniques with the applications in supply chain management , 2020, Int. J. Inf. Manag..

[23]  Xiangjun Zeng,et al.  Gearbox oil temperature anomaly detection for wind turbine based on sparse Bayesian probability estimation , 2020 .

[24]  Yu-Hsing Wang,et al.  Anomaly detection of defects on concrete structures with the convolutional autoencoder , 2020, Adv. Eng. Informatics.

[25]  L. Iocchi,et al.  A Close Look at Deep Learning with Small Data , 2020, 2020 25th International Conference on Pattern Recognition (ICPR).

[26]  Anahita Khojandi,et al.  Real-Time Sensor Anomaly Detection and Identification in Automated Vehicles , 2020, IEEE Transactions on Intelligent Transportation Systems.

[27]  A. P. Teixeira,et al.  Data mining approach to shipping route characterization and anomaly detection based on AIS data , 2020 .

[28]  B. Nachman,et al.  Anomaly detection with density estimation , 2020, Physical Review D.

[29]  Yaroslav M. Linder,et al.  Unsupervised Anomaly Detection in Time Series Using LSTM-Based Autoencoders , 2019, 2019 IEEE International Conference on Advanced Trends in Information Theory (ATIT).

[30]  Meng Wang,et al.  Generative Adversarial Active Learning for Unsupervised Outlier Detection , 2018, IEEE Transactions on Knowledge and Data Engineering.

[31]  Toby P. Breckon,et al.  GANomaly: Semi-Supervised Anomaly Detection via Adversarial Training , 2018, ACCV.

[32]  B. Ravi Kiran,et al.  An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos , 2018, J. Imaging.

[33]  Bayu Adhi Tama,et al.  Anomaly detection using random forest: A performance revisited , 2017, 2017 International Conference on Data and Software Engineering (ICoDSE).

[34]  Lovekesh Vig,et al.  LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detection , 2016, ArXiv.

[35]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[36]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[37]  Gang Wang,et al.  Deep Learning-Based Classification of Hyperspectral Data , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[38]  Jürgen Schmidhuber,et al.  Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction , 2011, ICANN.

[39]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[40]  Jürgen Schmidhuber,et al.  Learning to Forget: Continual Prediction with LSTM , 2000, Neural Computation.

[41]  David J. Sandoz,et al.  The application of principal component analysis and kernel density estimation to enhance process monitoring , 2000 .

[42]  S. Hochreiter,et al.  Long Short-Term Memory , 1997, Neural Computation.

[43]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[44]  Martin Fodslette Møller,et al.  A scaled conjugate gradient algorithm for fast supervised learning , 1993, Neural Networks.

[45]  Jianbo Yu,et al.  Convolutional Long Short-Term Memory Autoencoder-Based Feature Learning for Fault Detection in Industrial Processes , 2021, IEEE Transactions on Instrumentation and Measurement.

[46]  Anibal Flores,et al.  Data Augmentation for Short-Term Time Series Prediction with Deep Learning , 2021, SAI.

[47]  Xiaoping Shen,et al.  Kernel Density Estimation for An Anomaly Based Intrusion Detection System , 2006, MLMTA.