LSTM for Model-based Anomaly Detection in Cyber-Physical Systems

Anomaly detection is the task of detecting data which differs from the normal behaviour of a system in a given context. In order to approach this problem, data-driven models can be learned to predict current or future observations. Oftentimes, anomalous behaviour depends on the internal dynamics of the system and looks normal in a static context. To address this problem, the model should also operate depending on state. Long Short-Term Memory (LSTM) neural networks have been shown to be particularly useful to learn time sequences with varying length of temporal dependencies and are therefore an interesting general purpose approach to learn the behaviour of arbitrarily complex Cyber-Physical Systems. In order to perform anomaly detection, we slightly modify the standard norm 2 error to incorporate an estimate of model uncertainty. We analyse the approach on artificial and real data.

[1]  Oriol Vinyals,et al.  Bayesian Recurrent Neural Networks , 2017, ArXiv.

[2]  David A. Clifton,et al.  A review of novelty detection , 2014, Signal Process..

[3]  Justin Bayer,et al.  Variational Inference for On-line Anomaly Detection in High-Dimensional Time Series , 2016, ArXiv.

[4]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[5]  Oliver Niggemann,et al.  Symptomes Classifier Hypotheses Phenomenological Approach to Diagnosis Causality Analysis Causality Model Hypotheses Model-Based Approach to Diagnosis Symptomes Similarity Search Case Database Hypotheses , 2014 .

[6]  Lance Sherry,et al.  Anomaly detection in aircraft data using Recurrent Neural Networks (RNN) , 2016, 2016 Integrated Communications Navigation and Surveillance (ICNS).

[7]  Benno Stein,et al.  Learning Behavior Models for Hybrid Timed Systems , 2012, AAAI.

[8]  Lovekesh Vig,et al.  Long Short Term Memory Networks for Anomaly Detection in Time Series , 2015, ESANN.

[9]  Yarin Gal,et al.  Uncertainty in Deep Learning , 2016 .

[10]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[11]  Timothy J. O'Shea,et al.  Recurrent Neural Radio Anomaly Detection , 2016, ArXiv.

[12]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[13]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[14]  Shehroz S. Khan,et al.  One-class classification: taxonomy of study and review of techniques , 2013, The Knowledge Engineering Review.

[15]  Steven X. Ding,et al.  A Survey of Fault Diagnosis and Fault-Tolerant Techniques—Part II: Fault Diagnosis With Knowledge-Based and Hybrid/Active Approaches , 2015, IEEE Transactions on Industrial Electronics.

[16]  Maciej Wielgosz,et al.  Using LSTM recurrent neural networks for monitoring the LHC superconducting magnets , 2017 .

[17]  Nikolay Laptev,et al.  Deep and Confident Prediction for Time Series at Uber , 2017, 2017 IEEE International Conference on Data Mining Workshops (ICDMW).