Optimize the Coverage Probability of Prediction Interval for Anomaly Detection of Sensor-Based Monitoring Series

Effective anomaly detection of sensing data is essential for identifying potential system failures. Because they require no prior knowledge or accumulated labels, and provide uncertainty presentation, the probability prediction methods (e.g., Gaussian process regression (GPR) and relevance vector machine (RVM)) are especially adaptable to perform anomaly detection for sensing series. Generally, one key parameter of prediction models is coverage probability (CP), which controls the judging threshold of the testing sample and is generally set to a default value (e.g., 90% or 95%). There are few criteria to determine the optimal CP for anomaly detection. Therefore, this paper designs a graphic indicator of the receiver operating characteristic curve of prediction interval (ROC-PI) based on the definition of the ROC curve which can depict the trade-off between the PI width and PI coverage probability across a series of cut-off points. Furthermore, the Youden index is modified to assess the performance of different CPs, by the minimization of which the optimal CP is derived by the simulated annealing (SA) algorithm. Experiments conducted on two simulation datasets demonstrate the validity of the proposed method. Especially, an actual case study on sensing series from an on-orbit satellite illustrates its significant performance in practical application.

[1]  Sateesh Kumar Pradhan,et al.  ANOMALY DETECTION USING ARTIFICIAL NEURAL NETWORK , 2012 .

[2]  Slim Abdennadher,et al.  Enhancing one-class support vector machines for unsupervised anomaly detection , 2013, ODD '13.

[3]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevance Vector Machine , 2001 .

[4]  Chandan Srivastava,et al.  Support Vector Data Description , 2011 .

[5]  Quoc-Tuan Vien,et al.  On the Optimisation of Practical Wireless Indoor and Outdoor Microcells Subject to QoS Constraints , 2017 .

[6]  Yong Hu,et al.  The application of data mining techniques in financial fraud detection: A classification framework and an academic review of literature , 2011, Decis. Support Syst..

[7]  Tommy W. S. Chow,et al.  Self-Organizing and Self-Evolving Neurons: A New Neural Network for Optimization , 2007, IEEE Transactions on Neural Networks.

[8]  Jing Zhou,et al.  A Flexible Framework for Assessing the Quality of Information in Wireless Sensor Networks , 2015, Int. J. Distributed Sens. Networks.

[9]  Takehisa Yairi,et al.  An Anomaly Detection Method for Spacecraft Using Relevance Vector Learning , 2005, PAKDD.

[10]  Yu Peng,et al.  Anomaly detection of condition monitoring with predicted uncertainty for aerospace applications , 2015, 2015 12th IEEE International Conference on Electronic Measurement & Instruments (ICEMI).

[11]  Saeid Nahavandi,et al.  A prediction interval-based approach to determine optimal structures of neural network metamodels , 2010, Expert Syst. Appl..

[12]  Saeid Nahavandi,et al.  Construction of Optimal Prediction Intervals for Load Forecasting Problems , 2010, IEEE Transactions on Power Systems.

[13]  Eamonn J. Keogh,et al.  Finding surprising patterns in a time series database in linear time and space , 2002, KDD.

[14]  Saeid Nahavandi,et al.  Prediction Interval Construction and Optimization for Adaptive Neurofuzzy Inference Systems , 2011, IEEE Transactions on Fuzzy Systems.

[15]  Yu Peng,et al.  Anomaly detection based on data stream monitoring and prediction with improved Gaussian process regression algorithm , 2014, 2014 International Conference on Prognostics and Health Management.

[16]  David J. Hill,et al.  Anomaly detection in streaming environmental sensor data: A data-driven modeling approach , 2010, Environ. Model. Softw..

[17]  Borko Furht,et al.  Sensor fault and patient anomaly detection and classification in medical wireless sensor networks , 2013, 2013 IEEE International Conference on Communications (ICC).

[18]  Mia Hubert,et al.  Anomaly detection by robust statistics , 2017, WIREs Data Mining Knowl. Discov..

[19]  Tomoo Saito.  Bayesian anomaly detection in monitoring data applying relevance vector machine , 2011, Smart Structures and Materials + Nondestructive Evaluation and Health Monitoring.

[20]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[21]  Wei Xie,et al.  Fragment Anomaly Detection With Prediction and Statistical Analysis for Satellite Telemetry , 2017, IEEE Access.

[22]  Lin Li,et al.  Industrial communication intrusion detection algorithm based on improved one-class SVM , 2015, 2015 World Congress on Industrial Control Systems Security (WCICSS).

[23]  Yu Peng,et al.  Anomaly detection based on uncertainty fusion for univariate monitoring series , 2017 .

[24]  Robert P. W. Duin,et al.  Support Vector Data Description , 2004, Machine Learning.

[25]  M. Schlechtingen,et al.  Using Data-Mining Approaches for Wind Turbine Power Curve Monitoring: A Comparative Study , 2013, IEEE Transactions on Sustainable Energy.

[26]  Abbas Khosravi,et al.  Short-Term Load and Wind Power Forecasting Using Neural Network-Based Prediction Intervals , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[27]  J. Berger.  Statistical Decision Theory and Bayesian Analysis , 1988 .

[28]  Nozer D. Singpurwalla,et al.  Choosing a Coverage Probability for Prediction Intervals , 2008 .

[29]  Shinn-Ying Ho,et al.  Optimizing fuzzy neural networks for tuning PID controllers using an orthogonal simulated annealing algorithm OSA , 2006, IEEE Transactions on Fuzzy Systems.

[30]  Suleyman Serdar Kozat,et al.  Unsupervised Anomaly Detection With LSTM Neural Networks , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[31]  M. Tahar Kechadi,et al.  Grid-Based Approaches for Distributed Data Mining Applications , 2009, ArXiv.

[32]  Nezamabadi Pour Hossein,et al.  A Lattice based Nearest Neighbor Classifier for Anomaly Intrusion Detection , 2013 .

[33]  Xiaobing Wu,et al.  Approximation Algorithms for Cell Planning in Heterogeneous Networks , 2017, IEEE Transactions on Vehicular Technology.

[34]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[35]  Xiao-yun Chen,et al.  Multi-scale anomaly detection algorithm based on infrequent pattern of time series , 2008 .

[36]  Takahiro Saito,et al.  Semi-supervised Approach to Soft Sensor Modeling for Fault Detection in Industrial Systems with Multiple Operation Modes , 2017, AISI.

[37]  Michael E. Tipping.  Sparse Bayesian Learning and the Relevance Vector Machine , 2001, J. Mach. Learn. Res..

[38]  Yang Yanxi,et al.  Data series forecasting and anomaly detection methods based on online least squares support vector machine , 2013, Proceedings of the 32nd Chinese Control Conference.

[39]  Annie George,et al.  Anomaly Detection based on Machine Learning Dimensionality Reduction using PCA and Classification using SVM , 2012 .

[40]  Andrei V. Gurtov,et al.  A layered encryption mechanism for networked critical infrastructures , 2013, IEEE Network.

[41]  Fenxiong Chen,et al.  An Anomaly Detection Based on Data Fusion Algorithm in Wireless Sensor Networks , 2015, Int. J. Distributed Sens. Networks.

[42]  N. Perkins,et al.  The inconsistency of "optimal" cutpoints obtained using two criteria based on the receiver operating characteristic curve. , 2006, American journal of epidemiology.

[43]  Saeid Nahavandi,et al.  A New Fuzzy-Based Combined Prediction Interval for Wind Power Forecasting , 2016, IEEE Transactions on Power Systems.

[44]  Shengping Yang,et al.  The receiver operating characteristic (ROC) curve , 2017 .

[45]  Rajeev Kumar,et al.  Receiver operating characteristic (ROC) curve for medical researchers , 2011, Indian pediatrics.

[46]  P. Venkata Krishna,et al.  A Deep Learning Based Artificial Neural Network Approach for Intrusion Detection , 2017, ICMC.

[47]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[48]  A. Worster,et al.  Understanding receiver operating characteristic (ROC) curves. , 2006, CJEM.

[49]  Quoc-Tuan Vien,et al.  On the coverage and power allocation for downlink in heterogeneous wireless cellular networks , 2015, 2015 IEEE International Conference on Communications (ICC).

[50]  Derek S Young,et al.  Choosing a coverage probability for forecasting the incidence of cancer , 2014, Statistics in medicine.

[51]  Rachael Hamilton-Keene,et al.  Forecasting the Incidence of Cancer in Regional Victoria, Australia , 2013 .

[52]  Clement T. Yu,et al.  Haar Wavelets for Efficient Similarity Search of Time-Series: With and Without Time Warping , 2003, IEEE Trans. Knowl. Data Eng..