On-Line Error Detection and Mitigation for Time-Series Data of Cyber-Physical Systems using Deep Learning Based Methods

A cyber-physical system consists of sensors, micro-controller, networks, and actuators that interact with each other, generate a substantial amount of data, and form extremely complex system operational profiles. These heterogeneous components are subject to errors, e.g. spikes, off-sets, or delays, that may result in system failures. As the complexity of modern systems increases, it becomes a challenge to apply traditional fault detection and isolation methods to such complex systems. Deep learning based methods have surpassed traditional methods in terms of performance as the data size and complexity increase. The signals of cyber-physical systems are mainly time-series data. In this paper, we propose a new on-line error detection and mitigation approach for common sensor, computing hardware, and network errors of cyber-physical systems using deep learning based methods. More specifically, we train a Long Short-Term Memory (LSTM) network as a single step prediction model for the detection and mitigation of errors, like spikes, or offsets. In order to detect the long-duration errors that show no sharp change (a sudden drop or rise) between two successive data samples when errors occurred, e.g. network delays, we train an LSTM encoder-decoder as a multi-step prediction model. We also introduce the on-line error mitigation approach. Automatic recovery is achieved by replacing the detected errors with the predicted values. Finally, we demonstrate on-line error detection and mitigation capabilities of the trained single step and multi-step predictors using representative case studies.

[1]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[2]  Gregory J. Pottie,et al.  Sensor network data fault types , 2007, TOSN.

[3]  Henrik Eriksson,et al.  MODIFI: A MODel-Implemented Fault Injection Tool , 2010, SAFECOMP.

[4]  Antonio Pietrosanto,et al.  Instrument fault detection and isolation: state of the art and new research trends , 2000, IEEE Trans. Instrum. Meas..

[5]  Yu Peng,et al.  Review on cyber-physical systems , 2017, IEEE/CAA Journal of Automatica Sinica.

[6]  Carl E. Landwehr,et al.  Basic concepts and taxonomy of dependable and secure computing , 2004, IEEE Transactions on Dependable and Secure Computing.

[7]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[8]  Nikolay Laptev,et al.  Deep and Confident Prediction for Time Series at Uber , 2017, 2017 IEEE International Conference on Data Mining Workshops (ICDMW).

[9]  Dominique T. Shipmon,et al.  Time Series Anomaly Detection; Detection of anomalous drops with limited features and sparse examples in noisy highly periodic data , 2017, ArXiv.

[10]  Jasper Snoek,et al.  Practical Bayesian Optimization of Machine Learning Algorithms , 2012, NIPS.

[11]  Marios M. Polycarpou,et al.  Sensor Fault Diagnosis , 2016, Found. Trends Syst. Control..

[12]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[13]  Klaus Janschek,et al.  Reliability Evaluation of Functionally Equivalent Simulink Implementations of a PID Controller under Silent Data Corruption , 2018, 2018 IEEE 29th International Symposium on Software Reliability Engineering (ISSRE).

[14]  Akash Singh,et al.  Anomaly Detection for Temporal Data using Long Short-Term Memory (LSTM) , 2017 .

[15]  Suleyman Serdar Kozat,et al.  Unsupervised Anomaly Detection With LSTM Neural Networks , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Gerd Ascheid,et al.  Efficient On-Line Error Detection and Mitigation for Deep Neural Network Accelerators , 2018, SAFECOMP.

[17]  Lovekesh Vig,et al.  Anomaly detection in ECG time signals via deep long short-term memory networks , 2015, 2015 IEEE International Conference on Data Science and Advanced Analytics (DSAA).

[18]  Subutai Ahmad,et al.  Evaluating Real-Time Anomaly Detection Algorithms -- The Numenta Anomaly Benchmark , 2015, 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA).

[19]  Bora Caglayan,et al.  DeepAD: A Generic Framework Based on Deep Learning for Time Series Anomaly Detection , 2018, PAKDD.

[20]  Insoo Koo,et al.  Sensor Fault Classification Based on Support Vector Machine and Statistical Time-Domain Features , 2017, IEEE Access.

[21]  Lovekesh Vig,et al.  LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detection , 2016, ArXiv.

[22]  Valentino Constantinou,et al.  Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding , 2018, KDD.

[23]  Sanjay Chawla,et al.  Deep Learning for Anomaly Detection: A Survey , 2019, ArXiv.

[24]  Subutai Ahmad,et al.  Unsupervised real-time anomaly detection for streaming data , 2017, Neurocomputing.

[25]  Benjamin Lindemann,et al.  Anomaly detection in discrete manufacturing using self-learning approaches , 2019, Procedia CIRP.

[26]  Zhen Sun,et al.  Fault detection, isolation, and diagnosis of self-validating multifunctional sensors. , 2016, The Review of scientific instruments.

[27]  Li Jiang,et al.  Sensor Fault Detection and Isolation Using System Dynamics Identification Techniques. , 2011 .

[28]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[29]  Radu Marculescu,et al.  Multi-Scale Compositionality: Identifying the Compositional Structures of Social Dynamics Using Deep Learning , 2015, PloS one.

[30]  Charles C. Kemp,et al.  A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-Based Variational Autoencoder , 2017, IEEE Robotics and Automation Letters.

[31]  Lovekesh Vig,et al.  Long Short Term Memory Networks for Anomaly Detection in Time Series , 2015, ESANN.

[32]  Ramesh Govindan,et al.  Sensor faults: Detection methods and prevalence in real-world datasets , 2010, TOSN.