Internet Traffic Behavior Profiling for Network Security Monitoring

Recent spates of cyber-attacks and frequent emergence of applications affecting Internet traffic dynamics have made it imperative to develop effective techniques that can extract, and make sense of, significant communication patterns from Internet traffic data for use in network operations and security management. In this paper, we present a general methodology for building comprehensive behavior profiles of Internet backbone traffic in terms of communication patterns of end-hosts and services. Relying on data mining and entropy-based techniques, the methodology consists of significant cluster extraction, automatic behavior classification and structural modeling for in-depth interpretive analyses. We validate the methodology using data sets from the core of the Internet.

[1]  George Varghese,et al.  Automatically inferring patterns of resource consumption in network traffic , 2003, SIGCOMM '03.

[2]  Salvatore J. Stolfo,et al.  Behavior Profiling of Email , 2003, ISI.

[3]  Andrew B. Nobel,et al.  Statistical Clustering of Internet Communication Patterns , 2003 .

[4]  G. Klir,et al.  RECONSTRUCTABILITY ANALYSIS OF MULTI-DIMENSIONAL RELATIONS: A Theoretical Basis for Computer-Aided Determination of Acceptable Systems Models † , 1979 .

[5]  Zhi-Li Zhang,et al.  Reducing Unwanted Traffic in a Backbone Network , 2005, SRUTI.

[6]  Stuart Staniford-Chen,et al.  Practical Automated Detection of Stealthy Portscans , 2002, J. Comput. Secur..

[7]  Fang Hao,et al.  Real-time detection of hidden traffic patterns , 2004, Proceedings of the 12th IEEE International Conference on Network Protocols, 2004. ICNP 2004..

[8]  Hari Balakrishnan,et al.  Fast portscan detection using sequential hypothesis testing , 2004, IEEE Symposium on Security and Privacy, 2004. Proceedings. 2004.

[9]  Dong Xiang,et al.  Information-theoretic measures for anomaly detection , 2001, Proceedings 2001 IEEE Symposium on Security and Privacy. S&P 2001.

[10]  Divesh Srivastava,et al.  Finding Hierarchical Heavy Hitters in Data Streams , 2003, VLDB.

[11]  George C. Polyzos,et al.  A Parameterizable Methodology for Internet Traffic Flow Profiling , 1995, IEEE J. Sel. Areas Commun..

[12]  Mark Crovella,et al.  Mining anomalies using traffic feature distributions , 2005, SIGCOMM '05.

[13]  Martin Zwick,et al.  An overview of reconstructability analysis , 2004 .

[14]  Mark Crovella,et al.  Characterization of network-wide anomalies in traffic flows , 2004, IMC '04.

[15]  Michael I. Jordan.  Graphical Models , 2003 .

[16]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[17]  Michalis Faloutsos,et al.  BLINC: multilevel traffic classification in the dark , 2005, SIGCOMM '05.

[18]  Klaus Krippendorff,et al.  Information Theory: Structural Models for Qualitative Data. , 1988 .

[19]  Balachander Krishnamurthy,et al.  Flash crowds and denial of service attacks: characterization and implications for CDNs and web sites , 2002, WWW.

[20]  Robert K. Cunningham,et al.  A taxonomy of computer worms , 2003, WORM '03.

[21]  Carsten Lund,et al.  Online identification of hierarchical heavy hitters: algorithms, evaluation, and applications , 2004, IMC '04.

[22]  Mark Crovella,et al.  Diagnosing network-wide traffic anomalies , 2004, SIGCOMM '04.

[23]  Jaideep Srivastava,et al.  A Comparative Study of Anomaly Detection Schemes in Network Intrusion Detection , 2003, SDM.

[24]  Zhi-Li Zhang,et al.  Profiling internet backbone traffic: behavior models and applications , 2005, SIGCOMM '05.

[25]  Vinod Yegneswaran,et al.  Characteristics of internet background radiation , 2004, IMC '04.

[26]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[27]  Vinod Yegneswaran,et al.  Internet intrusions: global characteristics and prevalence , 2003, SIGMETRICS '03.

[28]  Balachander Krishnamurthy,et al.  Sketch-based change detection: methods, evaluation, and applications , 2003, IMC '03.

[29]  Matthew V. Mahoney,et al.  Network traffic anomaly detection based on packet bytes , 2003, SAC '03.

[30]  Zhi-Li Zhang,et al.  A Real-Time Network Traffic Profiling System , 2007, 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN'07).