Anomaly Detection for a Water Treatment System Using Unsupervised Machine Learning

In this paper, we propose and evaluate the application of unsupervised machine learning to anomaly detection for a Cyber-Physical System (CPS). We compare two methods: Deep Neural Networks (DNN) adapted to time series data generated by a CPS, and one-class Support Vector Machines (SVM). These methods are evaluated against data from the Secure Water Treatment (SWaT) testbed, a scaled-down but fully operational raw water purification plant. For both methods, we first train detectors using a log generated by SWaT operating under normal conditions. Then, we evaluate the performance of both methods using a log generated by SWaT operating under 36 different attack scenarios. We find that our DNN generates fewer false positives than our one-class SVM while our SVM detects slightly more anomalies. Overall, our DNN has a slightly better F measure than our SVM. We discuss the characteristics of the DNN and one-class SVM used in this experiment, and compare the advantages and disadvantages of the two methods.

[1]  Lovekesh Vig,et al.  Long Short Term Memory Networks for Anomaly Detection in Time Series , 2015, ESANN.

[2]  Hans-Peter Kriegel,et al.  LOF: identifying density-based local outliers , 2000, SIGMOD '00.

[3]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[4]  Sridhar Adepu,et al.  Distributed Detection of Single-Stage Multipoint Cyber Attacks in a Water Treatment Plant , 2016, AsiaCCS.

[5]  Sridhar Adepu,et al.  Anomaly Detection in Cyber Physical Systems Using Recurrent Neural Networks , 2017, 2017 IEEE 18th International Symposium on High Assurance Systems Engineering (HASE).

[6]  Osamu Mizuno,et al.  Log-Based Anomaly Detection of CPS Using a Statistical Method , 2017, 2017 8th International Workshop on Empirical Software Engineering in Practice (IWESEP).

[7]  Ali Alawi,et al.  Real Time Fault Diagnosis , 1997 .

[8]  Feng Zhao,et al.  Monitoring and fault diagnosis of hybrid systems , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[9]  Sridhar Adepu,et al.  Using Process Invariants to Detect Cyber Attacks on a Water Treatment System , 2016, SEC.

[10]  Kenta Oono,et al.  Chainer : a Next-Generation Open Source Framework for Deep Learning , 2015 .

[11]  Brian C. Williams,et al.  Hybrid estimation of complex systems , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[12]  Karl Henrik Johansson,et al.  Attack models and scenarios for networked control systems , 2012, HiCoNS '12.

[13]  Sridhar Adepu,et al.  A Dataset to Support Research in the Design of Secure Water Treatment Systems , 2016, CRITIS.

[14]  Jun Sun,et al.  Towards Learning and Verifying Invariants of Cyber-Physical Systems by Code Mutation , 2016, FM.

[15]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[16]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[17]  Thomas G. Dietterich Machine Learning for Sequential Data: A Review , 2002, SSPR/SPR.

[18]  Yu Cheng,et al.  Deep Structured Energy Based Models for Anomaly Detection , 2016, ICML.

[19]  Gautam Biswas,et al.  Model-Based Diagnosis of Hybrid Systems , 2003, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[20]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[21]  Sebastian Thrun,et al.  Real-time fault diagnosis [robot fault diagnosis] , 2004, IEEE Robotics & Automation Magazine.

[22]  Calin Belta,et al.  Anomaly detection in cyber-physical systems: A formal methods approach , 2014, 53rd IEEE Conference on Decision and Control.

[23]  Brian C. Williams,et al.  Mode Estimation of Probabilistic Hybrid Systems , 2002, HSCC.

[24]  Nils Ole Tippenhauer,et al.  HAMIDS: Hierarchical Monitoring Intrusion Detection System for Industrial Control Systems , 2016, CPS-SPC '16.

[25]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[26]  Florian Dörfler,et al.  Cyber-physical attacks in power networks: Models, fundamental limitations and monitor design , 2011, IEEE Conference on Decision and Control and European Control Conference.

[27]  Charu C. Aggarwal,et al.  Outlier Analysis , 2013, Springer New York.