A Transfer Learning Framework for Anomaly Detection Using Model of Normality

Convolutional Neural Network (CNN) techniques have proven to be very useful in image-based anomaly detection applications. CNN can be used as deep features extractor where other anomaly detection techniques are applied on these features. For this scenario, using transfer learning is common since pre-trained models provide deep feature representations that are useful for anomaly detection tasks. Consequentially, anomaly can be detected by applying similarly measure between extracted features and a defined model of normality. A key factor in such approaches is the decision threshold used for detecting anomaly. While most of the proposed methods focus on the approach itself, slight attention has been paid to address decision threshold settings. In this paper, we tackle this problem and propose a well-defined method to set the working-point decision threshold that improves detection accuracy. We introduce a transfer learning framework for anomaly detection based on similarity measure with a Model of Normality (MoN) and show that with the proposed threshold settings, a significant performance improvement can be achieved. Moreover, the framework has low complexity with relaxed computational requirements.

[1]  Aidong Zhang,et al.  Detecting ECG abnormalities via transductive transfer learning , 2012, BCB '12.

[2]  Michael Elad,et al.  Sparse Coding with Anomaly Detection , 2013, Journal of Signal Processing Systems.

[3]  Uma Boregowda,et al.  Similarity Based Feature Transformation for Network Anomaly Detection , 2020, IEEE Access.

[4]  Jiwen Lu,et al.  Deep transfer metric learning , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Fakhri Karray,et al.  Anomaly Detection for Images Using Auto-encoder Based Sparse Representation , 2020, ICIAR.

[6]  Abdallah Shami,et al.  Bayesian Optimization with Machine Learning Algorithms Towards Anomaly Detection , 2018, 2018 IEEE Global Communications Conference (GLOBECOM).

[7]  Fei-Fei Li,et al.  Online detection of unusual events in videos via dynamic sparse coding , 2011, CVPR 2011.

[8]  Abdallah Shami,et al.  Multi-Stage Optimized Machine Learning Framework for Network Intrusion Detection , 2020, IEEE Transactions on Network and Service Management.

[9]  Luc Van Gool,et al.  Detection and Identification of Rare Audiovisual Cues , 2012, Studies in Computational Intelligence.

[10]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[11]  Dorit Merhof,et al.  Modeling the Distribution of Normal Data in Pre-Trained Deep Features for Anomaly Detection , 2020, 2020 25th International Conference on Pattern Recognition (ICPR).

[12]  Jie Liu,et al.  Anomaly Detection in Manufacturing Systems Using Structured Neural Networks , 2018, 2018 13th World Congress on Intelligent Control and Automation (WCICA).

[13]  Giacomo Boracchi,et al.  Defect Detection in SEM Images of Nanofibrous Materials , 2017, IEEE Transactions on Industrial Informatics.

[14]  V Vaidehi,et al.  A transfer learning framework for traffic video using neuro-fuzzy approach , 2017, Sādhanā.

[15]  Quoc V. Le,et al.  EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks , 2019, ICML.

[16]  David A. Clifton,et al.  A review of novelty detection , 2014, Signal Process..

[17]  Naixue Xiong,et al.  Learning Sparse Representation With Variational Auto-Encoder for Anomaly Detection , 2018, IEEE Access.

[18]  Lewis D. Griffin,et al.  Transfer representation-learning for anomaly detection , 2016, ICML 2016.

[19]  Javier Villalba-Diez,et al.  Deep Learning for Industrial Computer Vision Quality Control in the Printing Industry 4.0 , 2019, Sensors.

[20]  Abdallah Shami,et al.  Distance-Based Anomaly Detection for Industrial Surfaces Using Triplet Networks , 2020, 2020 11th IEEE Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON).

[21]  Brendt Wohlberg,et al.  Novelty detection in images by sparse representations , 2014, 2014 IEEE Symposium on Intelligent Embedded Systems (IES).

[22]  David L. Neuhoff,et al.  Structural similarity metrics for texture analysis and retrieval , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[23]  Sheng Wang,et al.  BAT: Deep Learning Methods on Network Intrusion Detection Using NSL-KDD Dataset , 2020, IEEE Access.

[24]  Jin Wang,et al.  Anomaly Detection Based on Convolutional Recurrent Autoencoder for IoT Time Series , 2022, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[25]  Abdallah Shami,et al.  Data Mining Techniques in Intrusion Detection Systems: A Systematic Literature Review , 2018, IEEE Access.

[26]  Hanan Lutfiyya,et al.  DNS Typo-Squatting Domain Detection: A Data Analytics & Machine Learning Based Approach , 2018, 2018 IEEE Global Communications Conference (GLOBECOM).

[27]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[28]  P. Mahalanobis On the generalized distance in statistics , 1936 .

[29]  Bo Chen,et al.  MnasNet: Platform-Aware Neural Architecture Search for Mobile , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Paolo Napoletano,et al.  Anomaly Detection in Nanofibrous Materials by CNN-Based Self-Similarity , 2018, Sensors.

[31]  Abdallah Shami,et al.  Ensemble-based Feature Selection and Classification Model for DNS Typo-squatting Detection , 2020, 2020 IEEE Canadian Conference on Electrical and Computer Engineering (CCECE).

[32]  Xianbin Wang,et al.  Recursive Principal Component Analysis-Based Data Outlier Detection and Sensor Data Aggregation in IoT Systems , 2017, IEEE Internet of Things Journal.

[33]  Jean-Yves Tourneret,et al.  Anomaly detection in mixed telemetry data using a sparse representation and dictionary learning , 2020, Signal Process..

[34]  Junsong Yuan,et al.  Sparse reconstruction cost for abnormal event detection , 2011, CVPR 2011.

[35]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[36]  Zahir M. Hussain,et al.  An Entropy-Histogram Approach for Image Similarity and Face Recognition , 2018 .

[37]  Abdallah Shami,et al.  Tree-Based Intelligent Intrusion Detection System in Internet of Vehicles , 2019, 2019 IEEE Global Communications Conference (GLOBECOM).

[38]  H. B. Mitchell Image Fusion: Theories, Techniques and Applications , 2010 .

[39]  Brett J. Borghetti,et al.  A Survey of Distance and Similarity Measures Used Within Network Intrusion Anomaly Detection , 2015, IEEE Communications Surveys & Tutorials.

[40]  Quoc V. Le,et al.  Do Better ImageNet Models Transfer Better? , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  David Zhang,et al.  A Survey of Sparse Representation: Algorithms and Applications , 2015, IEEE Access.

[42]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[43]  Peter Christiansen,et al.  DeepAnomaly: Combining Background Subtraction and Deep Learning for Detecting Obstacles and Anomalies in an Agricultural Field , 2016, Sensors.

[44]  Hai-Dong Yang,et al.  Transfer learning for aluminium extrusion electricity consumption anomaly detection via deep neural networks , 2017, Int. J. Comput. Integr. Manuf..

[45]  Michael Lütjen,et al.  Anomaly detection with convolutional neural networks for industrial surface inspection , 2019, Procedia CIRP.

[46]  Raghavendra Chalapathy University of Sydney,et al.  Deep Learning for Anomaly Detection: A Survey , 2019, ArXiv.

[47]  Jie Liu,et al.  Semi-supervised anomaly detection with dual prototypes autoencoder for industrial surface inspection , 2021 .