An unsupervised anomaly detection approach using energy-based spatiotemporal graphical modeling

Abstract This paper presents a new data-driven framework for unsupervised system-wide anomaly detection for modern distributed complex systems within which there exists a strong connectivity among sub-systems, operating in diverse modes and encountering a large variety of anomalies. The framework is based on a spatiotemporal feature extraction scheme built on the concept of symbolic dynamics for discovering and representing causal interactions among subsystems. The extracted features from the spatiotemporal pattern network (STPN) are then used to learn system-wide patterns via a Restricted Boltzmann Machine (RBM), to form an energy based anomaly detection approach. While STPN is treated as a weak learner of system modes (in terms of difficulty in discovering true graphical representations), RBM is treated as a boosting approach to form a strong learner of system characteristics. Case studies with simulated data and real data from an integrated building system are used to validate the proposed approach. The results show that: (i) the increase in RBM free energy in the off-nominal conditions compared to that in the nominal conditions can be used for anomaly detection; (ii) the proposed framework formulates a strong learning model (STPN+RBM) from weak frequentist model–STPN, via boosting with RBM; and (iii) the STPN+RBM framework can capture multiple nominal operating modes of distributed complex systems with a single graphical model.

[1]  Christopher Leckie,et al.  High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning , 2016, Pattern Recognit..

[2]  Ishanu Chattopadhyay,et al.  Causality Networks , 2014, ArXiv.

[3]  Jochen Kaiser,et al.  Transfer entropy in magnetoencephalographic data: quantifying information flow in cortical and cerebellar networks. , 2011, Progress in biophysics and molecular biology.

[4]  Kai Liu,et al.  Adaptive fuzzy clustering based anomaly data detection in energy system of steel industry , 2014, Inf. Sci..

[5]  Chao Liu,et al.  Data driven exploration of traffic network system dynamics using high resolution probe data , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[6]  Jan Kleissl,et al.  Cyber-physical energy systems: Focus on smart buildings , 2010, Design Automation Conference.

[7]  Geoffrey E. Hinton,et al.  An Efficient Learning Procedure for Deep Boltzmann Machines , 2012, Neural Computation.

[8]  Yan Liu,et al.  Granger Causality for Time-Series Anomaly Detection , 2012, 2012 IEEE 12th International Conference on Data Mining.

[9]  Joel J. P. C. Rodrigues,et al.  A seven-dimensional flow analysis to help autonomous network management , 2014, Inf. Sci..

[10]  Tijmen Tieleman,et al.  Training restricted Boltzmann machines using approximations to the likelihood gradient , 2008, ICML '08.

[11]  Kushal Mukherjee,et al.  Multi-sensor information fusion for fault detection in aircraft gas turbine engines , 2013 .

[12]  Hector Garcia-Molina,et al.  Web graph similarity for anomaly detection , 2010, Journal of Internet Services and Applications.

[13]  Chao Liu,et al.  Machine Condition Classification Using Deterioration Feature Extraction and Anomaly Determination , 2011, IEEE Transactions on Reliability.

[14]  Chao Liu,et al.  Global geometric similarity scheme for feature selection in fault diagnosis , 2014, Expert Syst. Appl..

[15]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[16]  Thomas S. Richardson,et al.  A Discovery Algorithm for Directed Cyclic Graphs , 1996, UAI.

[17]  Abhishek Srivastav,et al.  A composite discretization scheme for symbolic identification of complex systems , 2016, Signal Process..

[18]  Rainer Goebel,et al.  Investigating directed cortical interactions in time-resolved fMRI data using vector autoregressive modeling and Granger causality mapping. , 2003, Magnetic resonance imaging.

[19]  Jon Williamson,et al.  Causality and Probability in the Sciences , 2007 .

[20]  Bernhard Schölkopf,et al.  On Causal Discovery with Cyclic Additive Noise Models , 2011, NIPS.

[21]  Asok Ray,et al.  Review and comparative evaluation of symbolic dynamic filtering for detection of anomaly patterns , 2009, 2008 American Control Conference.

[22]  Asok Ray,et al.  An inner product space on irreducible and synchronizable probabilistic finite state automata , 2012, Mathematics of Control, Signals, and Systems.

[23]  Frederick Eberhardt,et al.  Learning linear cyclic causal models with latent variables , 2012, J. Mach. Learn. Res..

[24]  Nicolas Le Roux,et al.  Representational Power of Restricted Boltzmann Machines and Deep Belief Networks , 2008, Neural Computation.

[25]  Xin Jin,et al.  Optimization of symbolic feature extraction for pattern classification , 2012, Signal Process..

[26]  Asok Ray,et al.  Fault detection and isolation in aircraft gas turbine engines. Part 1: Underlying concept , 2008 .

[27]  Huaiyu Zhu.  On Information and Sufficiency , 1997 .

[28]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[29]  Abhishek Srivastav,et al.  Maximally Bijective Discretization for data-driven modeling of complex systems , 2013, 2013 American Control Conference.

[30]  Soumik Sarkar,et al.  Scalable Anomaly Detection and Isolation in Cyber-Physical Systems Using Bayesian Networks , 2014 .

[31]  Bin Fang,et al.  A novel item anomaly detection approach against shilling attacks in collaborative recommendation systems using the dynamic time interval segmentation technique , 2015, Inf. Sci..

[32]  RayAsok.  Symbolic dynamic analysis of complex systems for anomaly detection , 2004 .

[33]  Kushal Mukherjee,et al.  Generalization of Hilbert transform for symbolic analysis of noisy signals , 2009, Signal Process..

[34]  R. Scheines,et al.  Interventions and Causal Inference , 2007, Philosophy of Science.

[35]  Edwin Lughofer,et al.  Fuzzy fault isolation using gradient information and quality criteria from system identification models , 2015, Inf. Sci..

[36]  Matthew B Kennel,et al.  Statistically relaxing to generating partitions for observed time-series data. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[37]  Brendan J. Frey,et al.  A comparison of algorithms for inference and learning in probabilistic graphical models , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Robert E. Schapire,et al.  The Boosting Approach to Machine Learning An Overview , 2003 .

[39]  Liviu Miclea,et al.  Cyber-Physical Systems - Concept, Challenges and Research Areas , 2012 .

[40]  Asok Ray,et al.  Sensor Fusion for Fault Detection and Classification in Distributed Physical Processes , 2014, Front. Robot. AI.

[41]  Sinh Hoa Nguyen,et al.  On Finding Optimal Discretizations for Two Attributes , 1998, Rough Sets and Current Trends in Computing.

[42]  Siddharth Sridhar,et al.  Cyber–Physical System Security for the Electric Power Grid , 2012, Proceedings of the IEEE.

[43]  Munther A. Dahleh,et al.  Structure learning in causal cyclic networks , 2008 .

[44]  Jugal K. Kalita,et al.  A multi-step outlier-based anomaly detection approach to network-wide traffic , 2016, Inf. Sci..

[45]  Victor Solo,et al.  On causality and mutual information , 2008, 2008 47th IEEE Conference on Decision and Control.

[46]  Hiroki Takakura,et al.  Toward a more practical unsupervised anomaly detection system , 2013, Inf. Sci..

[47]  Chao Liu,et al.  Bridge damage detection using spatiotemporal patterns extracted from dense sensor network , 2016 .

[48]  Chao Liu,et al.  An Unsupervised Spatiotemporal Graphical Modeling Approach to Anomaly Detection in Distributed CPS , 2016, 2016 ACM/IEEE 7th International Conference on Cyber-Physical Systems (ICCPS).

[49]  Patrik O. Hoyer,et al.  Discovering Cyclic Causal Models by Independent Components Analysis , 2008, UAI.

[50]  Asok Ray,et al.  Symbolic dynamic analysis of complex systems for anomaly detection , 2004, Signal Process..

[51]  Gregory Gutin,et al.  Digraphs - theory, algorithms and applications , 2002 .