IP Network Anomaly Detection using Machine Learning

The proliferation of network technologies and its associated threats have made it indispensable to develop different techniques to effectively detect network attacks. The present paper focuses on discovering viable anomalies that have a significant potential to be associated with abnormal network behavior. Three approaches based on ML (Machine Learning) have been proposed to detect suspicious network behavior. These methods are an extension of the techniques discussed as part of the introduction below. The developed approaches in the current paper are evaluated by testing their efficiency against a real-time network attack using available open-source network tools. The results of the experiment demonstrate successful identification of anomalous instances from the telemetry data with a low false alarm rate. Further, we believe that our approaches can be directly deployed in a real-time environment (independently on the edge device or over the cloud) to strengthen the network security.

[1]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[2]  Lovekesh Vig,et al.  Long Short Term Memory Networks for Anomaly Detection in Time Series , 2015, ESANN.

[3]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[4]  David Heckerman,et al.  A Tutorial on Learning with Bayesian Networks , 1999, Innovations in Bayesian Networks.

[5]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[6]  Michèle Basseville,et al.  Detection of abrupt changes: theory and application , 1993 .

[7]  Hamid H. Jebur,et al.  Machine Learning Techniques for Anomaly Detection: An Overview , 2013 .

[8]  Teuvo Kohonen,et al.  The self-organizing map , 1990, Neurocomputing.

[9]  J. C. Dunn,et al.  A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters , 1973 .

[10]  Mark A. Girolami,et al.  An empirical analysis of the probabilistic K-nearest neighbour classifier , 2007, Pattern Recognit. Lett..

[11]  Gulshan Kumar,et al.  The use of artificial intelligence based techniques for intrusion detection: a review , 2010, Artificial Intelligence Review.

[12]  Tanja Zseby,et al.  Analysis of network traffic features for anomaly detection , 2014, Machine Learning.

[13]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[14]  Thomas S. Ferguson,et al.  On the Rejection of Outliers , 1961 .

[15]  T. Sargent,et al.  The multivariate normal distribution , 1989 .