Robust regression for anomaly detection

In our previous work, we have applied ordinary linear regression equation to network anomaly detection. However, the performance of ordinary linear regression equation is susceptible to outliers. Unfortunately, it is almost impossible to obtain a “clean” traffic data set for ordinary regression model due to the burstiness of network traffic and the pervasive network attacks. In this paper, we make use of robust regression techniques to mitigate the impact of outliers in the training data set. The experiment results show that the robust regression based method is more reliable than the ordinary regression based method in the face of outliers.

[1]  Yong Guan,et al.  Cardinality change-based early detection of large-scale cyber-attacks , 2013, 2013 Proceedings IEEE INFOCOM.

[2]  Ramesh Govindan,et al.  ASTUTE: detecting a different class of traffic anomalies , 2010, SIGCOMM '10.

[3]  Daojing He,et al.  Network Anomaly Detection Using Unsupervised Feature Selection and Density Peak Clustering , 2016, ACNS.

[4]  Jennifer Rexford,et al.  Sensitivity of PCA for traffic anomaly detection , 2007, SIGMETRICS '07.

[5]  Mark Crovella,et al.  Mining anomalies using traffic feature distributions , 2005, SIGCOMM '05.

[6]  Fuliang Li,et al.  A New Anomaly Detection Method Based on IGTE and IGFE , 2014, SecureComm.

[7]  Peter J. Rousseeuw,et al.  Robust Regression and Outlier Detection , 2005, Wiley Series in Probability and Statistics.

[8]  Christophe Diot,et al.  Diagnosing network-wide traffic anomalies , 2004, SIGCOMM.

[9]  Jianping Wu,et al.  CNGI-CERNET2: an IPv6 deployment in China , 2011, CCRV.

[10]  Dan Pei,et al.  Opprentice: Towards Practical and Automatic Anomaly Detection Through Machine Learning , 2015, Internet Measurement Conference.

[11]  Sean Hughes,et al.  Clustering by Fast Search and Find of Density Peaks , 2016 .

[12]  Ling Huang,et al.  Stealthy poisoning attacks on PCA-based anomaly detectors , 2009, SIGMETRICS Perform. Evaluation Rev..

[13]  Sonia Fahmy,et al.  Pegasus: Precision hunting for icebergs and anomalies in network flows , 2013, 2013 Proceedings IEEE INFOCOM.

[14]  Donald F. Towsley,et al.  Detecting anomalies in network traffic using maximum entropy estimation , 2005, IMC '05.

[15]  Fabio Soldo,et al.  Traffic anomaly detection based on the IP size distribution , 2012, 2012 Proceedings IEEE INFOCOM.