Sparse Representation and Dictionary Learning for Network Traffic Anomaly Detection

In this article we present the use of sparse representation of signal and dictionary learning method for solving the anomaly detection problem. The signals analysed in the article represented selected features of the network traffic. In the learning process we used modified Method of Optimal Directions in order to find a dictionary resembling correct structures of the network traffic deprived of the influence of possible outlying observations (outliers). A dictionary defined in such a way constituted basis for sparse representation of the analysed signal. Anomaly detection is realised by parameter estimation of the analysed signal and its comparative analysis to network traffic profiles. Efficiency of our method is examined with the use of extended set of test traces from real network traffic. The received experimental results confirm effectiveness of the presented method.

[1]  Michael Elad,et al.  Sparse Coding with Anomaly Detection , 2013, Journal of Signal Processing Systems.

[2]  Heung-No Lee,et al.  Reduced Computational Complexity Orthogonal Matching Pursuit Using a Novel Partitioned Inversion Technique for Compressive Sensing , 2018 .

[3]  Rémi Gribonval,et al.  Dictionary identifiability from few training samples , 2008, 2008 16th European Signal Processing Conference.

[4]  Victoria J. Hodge,et al.  A Survey of Outlier Detection Methodologies , 2004, Artificial Intelligence Review.

[5]  Tomasz Andrysiak,et al.  Anomaly Detection Preprocessor for SNORT IDS System , 2012, IP&C.

[6]  Kjersti Engan,et al.  Method of optimal directions for frame design , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[7]  S. Mallat,et al.  Adaptive greedy approximations , 1997 .

[8]  Gabriel Maciá-Fernández,et al.  Anomaly-based network intrusion detection: Techniques, systems and challenges , 2009, Comput. Secur..

[9]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[10]  Michael Elad,et al.  Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing , 2010 .

[11]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[12]  Fabio Roli,et al.  Security Evaluation of Pattern Classifiers under Attack , 2014, IEEE Transactions on Knowledge and Data Engineering.

[13]  Antonio Pescapè,et al.  NIS04-1: Wavelet-based Detection of DoS Attacks , 2006, IEEE Globecom 2006.

[14]  Felix Naumann,et al.  Data fusion , 2009, CSUR.

[15]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[16]  Michael Elad,et al.  From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images , 2009, SIAM Rev..

[17]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Alberto Dainotti,et al.  Wavelet-based Detection of DoS Attacks. , 2006 .

[19]  S.Y. Lim,et al.  Network Anomaly Detection System: The State of Art of Network Behaviour Analysis , 2008, 2008 International Conference on Convergence and Hybrid Information Technology.

[20]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[21]  Philippe Owezarski,et al.  Non-Gaussian and Long Memory Statistical Characterizations for Internet Traffic with Anomalies , 2007, IEEE Transactions on Dependable and Secure Computing.

[22]  Zhihua Lai,et al.  The Impact of Antenna Height on 3D Channel: A Ray Launching Based Analysis , 2018 .

[23]  Orhan Gazi.  Understanding Digital Signal Processing , 2017 .

[24]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[25]  Michael Elad,et al.  Dictionaries for Sparse Representation Modeling , 2010, Proceedings of the IEEE.

[26]  Christian Jutten,et al.  Outlier-aware dictionary learning for sparse representation , 2014, 2014 IEEE International Workshop on Machine Learning for Signal Processing (MLSP).

[27]  Vipin Kumar,et al.  Anomaly Detection for Discrete Sequences: A Survey , 2012, IEEE Transactions on Knowledge and Data Engineering.