Learning Entropy

Entropy has been widely used for anomaly detection in various disciplines. One such is in network attack detection, where its role is to detect significant changes in underlying distribution shape due to anomalous behaviour such as attacks. In this paper, we point out that entropy has significant blind spots, which can be made use by adversaries to evade detection. To illustrate the potential pitfalls, we give an in-principle analysis of network attack detection, in which we design a camouflage technique and show analytically that it can perfectly mask attacks from entropy based detector with low costs in terms of the volume of traffic brought in for camouflage. Finally, we illustrate and apply our technique to both synthetic distributions and ones taken from real traffic traces, and show how attacks undermine the detector.

[1]  Dan Schnackenberg,et al.  Statistical approaches to DDoS attack detection and response , 2003, Proceedings DARPA Information Survivability Conference and Exposition.

[2]  Pedro M. Domingos,et al.  Adversarial classification , 2004, KDD.

[3]  Artur Ziviani,et al.  Network anomaly detection using nonextensive entropy , 2007, IEEE Communications Letters.

[4]  Didier Sornette,et al.  Beyond Shannon: Characterizing Internet Traffic with Generalized Entropy Metrics , 2009, PAM.

[5]  Ling Huang,et al.  ANTIDOTE: understanding and defending against poisoning of anomaly detectors , 2009, IMC '09.

[6]  Christopher Meek,et al.  Adversarial learning , 2005, KDD '05.

[7]  Dong Xiang,et al.  Information-theoretic measures for anomaly detection , 2001, Proceedings 2001 IEEE Symposium on Security and Privacy. S&P 2001.

[8]  Jianying Zhou,et al.  DDoS Attack Detection Algorithms Based on Entropy Computing , 2007, ICICS.

[9]  Vyas Sekar,et al.  An empirical evaluation of entropy-based traffic anomaly detection , 2008, IMC '08.

[10]  Jianying Zhou,et al.  Information and Communications Security , 2013, Lecture Notes in Computer Science.

[11]  Bernhard Plattner,et al.  Entropy based worm and anomaly detection in fast IP networks , 2005, 14th IEEE International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprise (WETICE'05).

[12]  Mehmet Celenk,et al.  Anomaly detection and visualization using Fisher Discriminant clustering of network entropy , 2008, 2008 Third International Conference on Digital Information Management.