Protection of Superconducting Industrial Machinery Using RNN-Based Anomaly Detection for Implementation in Smart Sensor †

Sensing the voltage developed over a superconducting object is very important in order to make superconducting installation safe. An increase in the resistive part of this voltage (quench) can lead to significant deterioration or even to the destruction of the superconducting device. Therefore, detection of anomalies in time series of this voltage is mandatory for reliable operation of superconducting machines. The largest superconducting installation in the world is the main subsystem of the Large Hadron Collider (LHC) accelerator. Therefore a protection system was built around superconducting magnets. Currently, the solutions used in protection equipment at the LHC are based on a set of hand-crafted custom rules. They were proved to work effectively in a range of applications such as quench detection. However, these approaches lack scalability and require laborious manual adjustment of working parameters. The presented work explores the possibility of using the embedded Recurrent Neural Network as a part of a protection device. Such an approach can scale with the number of devices and signals in the system, and potentially can be automatically configured to given superconducting magnet working conditions and available data. In the course of the experiments, it was shown that the model using Gated Recurrent Units (GRU) comprising of two layers with 64 and 32 cells achieves 0.93 accuracy for anomaly/non-anomaly classification, when employing custom data compression scheme. Furthermore, the compression of proposed module was tested, and showed that the memory footprint can be reduced four times with almost no performance loss, making it suitable for hardware implementation.

[1]  Jianmin Jiang,et al.  One class support vector machine for anomaly detection in the communication network performance data , 2007 .

[2]  Alex Graves,et al.  Supervised Sequence Labelling with Recurrent Neural Networks , 2012, Studies in Computational Intelligence.

[3]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[4]  Mykel J. Kochenderfer,et al.  Analysis of Recurrent Neural Networks for Probabilistic Modeling of Driver Behavior , 2017, IEEE Transactions on Intelligent Transportation Systems.

[5]  Vipin Kumar,et al.  Comparative Evaluation of Anomaly Detection Techniques for Sequence Data , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[6]  Yongqiang Wang,et al.  Efficient Training and Evaluation of Recurrent Neural Network Language Models for Automatic Speech Recognition , 2016, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[7]  David A. Clifton,et al.  A review of novelty detection , 2014, Signal Process..

[8]  Andrzej Skoczeń,et al.  Design of FPGA-based radiation tolerant quench detectors for LHC , 2017 .

[9]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[10]  Maciej Wielgosz,et al.  The model of an anomaly detector for HiLumi LHC magnets based on Recurrent Neural Networks and adaptive quantization , 2018, Eng. Appl. Artif. Intell..

[11]  Network behaviour anomaly detection using Holt-Winters algorithm , 2011, 2011 International Conference for Internet Technology and Secured Transactions.

[12]  Vanish Talwar,et al.  Statistical techniques for online anomaly detection in data centers , 2011, 12th IFIP/IEEE International Symposium on Integrated Network Management (IM 2011) and Workshops.

[13]  Understanding Mahalanobis distance criterion for feature selection , 2015 .

[14]  Wonyong Sung,et al.  Fixed-point performance analysis of recurrent neural networks , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[15]  Roberto Hornero,et al.  Spectral and Nonlinear Analyses of MEG Background Activity in Patients With Alzheimer's Disease , 2008, IEEE Transactions on Biomedical Engineering.

[16]  Maciej Wielgosz,et al.  Looking for a Correct Solution of Anomaly Detection in the LHC Machine Protection System , 2018, 2018 International Conference on Signals and Electronic Systems (ICSES).

[17]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[18]  Vivienne Sze,et al.  Hardware for machine learning: Challenges and opportunities , 2017, 2017 IEEE Custom Integrated Circuits Conference (CICC).

[19]  Maciej Wielgosz,et al.  Using LSTM recurrent neural networks for detecting anomalous behavior of LHC superconducting magnets , 2016, ArXiv.

[20]  Jian Cheng,et al.  Quantized Convolutional Neural Networks for Mobile Devices , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  J. Ma,et al.  Time-series novelty detection using one-class support vector machines , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[22]  Zhi-Hua Zhou,et al.  Isolation Forest , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[23]  Bernhard Schölkopf,et al.  Support Vector Method for Novelty Detection , 1999, NIPS.

[24]  Ruben Ruiz-Gonzalez,et al.  An SVM-Based Classifier for Estimating the State of Various Rotating Components in Agro-Industrial Machinery with a Vibration Signal Acquired from a Single Point on the Machine Chassis , 2014, Sensors.

[25]  R. Denz,et al.  Electronic Systems for the Protection of Superconducting Elements in the LHC , 2006, IEEE Transactions on Applied Superconductivity.

[26]  Shuchang Zhou,et al.  Effective Quantization Methods for Recurrent Neural Networks , 2016, ArXiv.

[27]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[28]  Hojjat Salehinejad,et al.  Recurrent Neural Networks for Sequential Phenotype Prediction in Genomics , 2015, 2015 International Conference on Developments of E-Systems Engineering (DeSE).

[29]  Daowei Liu,et al.  Anomaly Detection of Single Sensors Using OCSVM_KNN , 2015, BigCom.

[30]  John Salvatier,et al.  Theano: A Python framework for fast computation of mathematical expressions , 2016, ArXiv.