Impact of Histogram Construction Techniques on Information - Theoretic Anomaly Detection

Thanks to its ability to face unknown attacks, anomaly-based intrusion detection is a key research topic in network security. In this paper anomalies are addressed from an Information theory perspective: in a nutshell, it is assumed that attacks determine a significant change in the distribution of relevant traffic descriptors and this change is measured in terms of Shannon entropy. In more detail, the traffic is first aggregated by means of random data structures (namely three-dimensional reversible sketches) and then the entropy associated to different traffic descriptors (for sake of brevity, we focus on the numbers of flows and bytes) is computed by using two alternative constructions of the corresponding empirical distributions, one based on the flows destination address and the other on their volume. The experimental results obtained over the MAWILab dataset validate the system and demonstrate the relevance of the way in which the histogram is built.

[1]  Yan Chen,et al.  Reversible sketches for efficient and accurate change detection over network data streams , 2004, IMC '04.

[2]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[3]  Mikkel Thorup,et al.  Tabulation based 4-universal hashing with applications to second moment estimation , 2004, SODA '04.

[4]  V. Sangeetha,et al.  Entropy based Anomaly Detection System to Prevent DDoS Attacks in Cloud , 2013, ArXiv.

[5]  Balachander Krishnamurthy,et al.  Sketch-based change detection: methods, evaluation, and applications , 2003, IMC '03.

[6]  Bernhard Plattner,et al.  Entropy based worm and anomaly detection in fast IP networks , 2005, 14th IEEE International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprise (WETICE'05).

[7]  Christian Callegari,et al.  On the Use of Compression Algorithms for Network Anomaly Detection , 2009, 2009 IEEE International Conference on Communications.

[8]  Christian Callegari,et al.  When randomness improves the anomaly detection performance , 2010, 2010 3rd International Symposium on Applied Sciences in Biomedical and Communication Technologies (ISABEL 2010).

[9]  Christophe Diot,et al.  Diagnosing network-wide traffic anomalies , 2004, SIGCOMM.

[10]  Christian Callegari,et al.  Entropy-based network anomaly Detection , 2017, 2017 International Conference on Computing, Networking and Communications (ICNC).

[11]  Lele Zhang,et al.  Learning Entropy , 2011, Networking.

[12]  Graham Cormode,et al.  An improved data stream summary: the count-min sketch and its applications , 2004, J. Algorithms.

[13]  Michele Colajanni,et al.  Evaluation of anomaly detection for in-vehicle networks through information-theoretic algorithms , 2016, 2016 IEEE 2nd International Forum on Research and Technologies for Society and Industry Leveraging a better tomorrow (RTSI).

[14]  Dong Xiang,et al.  Information-theoretic measures for anomaly detection , 2001, Proceedings 2001 IEEE Symposium on Security and Privacy. S&P 2001.

[15]  Mahdi Abadi,et al.  DroidMalHunter: A novel entropy-based anomaly detection system to detect malicious Android applications , 2015, 2015 5th International Conference on Computer and Knowledge Engineering (ICCKE).

[16]  Christian Callegari,et al.  Sketch-based multidimensional IDS: A new approach for network anomaly detection , 2013, 2013 IEEE Conference on Communications and Network Security (CNS).

[17]  Benoit Claise,et al.  Cisco Systems NetFlow Services Export Version 9 , 2004, RFC.