Network intrusion detection: monitoring, simulation and visualization

This dissertation presents our work on the network intrusion detection and intrusion simulation. The work in intrusion detection consists of two different network anomaly-based approaches. The work in intrusion simulation introduces a model using explicit traffic generation for the packet level traffic simulation. The process of anomaly detection is to first build profiles for the normal network activity and then mark any events or activities that deviate from the normal profiles as suspicious. Based on the different schemes of creating the normal activity profiles, we introduce two approaches for intrusion detection. The first one is a frequency-based approach which creates a normal frequency profile based on the periodical patterns existed in the time-series formed by the traffic. It aims at those attacks that are conducted by running pre-written scripts, which automate the process of attempting connections to various ports or sending packets with fabricated payloads, etc. The second approach builds the normal profile based on variations of connection-based behavior of each single computer. The deviations resulted from each individual computer are carried out by a weight assignment scheme and further used to build a weighted link graph representing the overall traffic abnormalities. The functionality of this system is of a distributed personal IDS system that also provides a centralized traffic analysis by graphical visualization. It provides a finer control over the internal network by focusing on connection-based behavior of each single computer. For network intrusion simulation, we explore an alternative method for network traffic simulation using explicit traffic generation. In particular, we build a model to replay the standard DARPA traffic data or the traffic data captured from a real environment. The replayed traffic data is mixed with the attacks, such as DOS and Probe attack, which can create apparent abnormal traffic flow patterns. With the explicit traffic generation, every packet that has ever been sent by the victim and attacker is formed in the simulation model and travels around strictly following the criteria of time and path that extracted from the real scenario. Thus, the model provides a promising aid in the study of intrusion detection techniques.

[1]  Philip K. Chan,et al.  Learning nonstationary models of normal network traffic for detecting novel attacks , 2002, KDD.

[2]  Biswanath Mukherjee,et al.  A network security monitor , 1990, Proceedings. 1990 IEEE Computer Society Symposium on Research in Security and Privacy.

[3]  David G. Andersen,et al.  Proceedings of Usits '03: 4th Usenix Symposium on Internet Technologies and Systems Mayday: Distributed Filtering for Internet Services , 2022 .

[4]  Xiangyang Li,et al.  Decision Tree Classifiers for Computer Intrusion Detection , 2001, Scalable Comput. Pract. Exp..

[5]  Li Jun,et al.  HIDE: a Hierarchical Network Intrusion Detection System Using Statistical Preprocessing and Neural Network Classification , 2001 .

[6]  Tao Wan,et al.  IntruDetector: a software platform for testing network intrusion detection algorithms , 2001, Seventeenth Annual Computer Security Applications Conference.

[7]  Barak A. Pearlmutter,et al.  Detecting intrusions using system calls: alternative data models , 1999, Proceedings of the 1999 IEEE Symposium on Security and Privacy (Cat. No.99CB36344).

[8]  Wenke Lee,et al.  Proactive detection of distributed denial of service attacks using MIB traffic variables-a feasibility study , 2001, 2001 IEEE/IFIP International Symposium on Integrated Network Management Proceedings. Integrated Network Management VII. Integrated Management Strategies for the New Millennium (Cat. No.01EX470).

[9]  Salvatore J. Stolfo,et al.  Adaptive Intrusion Detection: A Data Mining Approach , 2000, Artificial Intelligence Review.

[10]  Jim Alves-Foss,et al.  NATE: Network Analysis of Anomalous Traffic Events, a low-cost approach , 2001, NSPW '01.

[11]  Oliver Niggemann,et al.  Supporting Intrusion Detection by Graph Clustering and Graph Drawing , 2000 .

[12]  Deborah A. Frincke,et al.  Intrusion and Misuse Detection in Large-Scale Systems , 2002, IEEE Computer Graphics and Applications.

[13]  H. T. Kung,et al.  Use of spectral analysis in defense against DoS attacks , 2002, Global Telecommunications Conference, 2002. GLOBECOM '02. IEEE.

[14]  Matthew V. Mahoney,et al.  Network traffic anomaly detection based on packet bytes , 2003, SAC '03.

[15]  David Moore,et al.  Code-Red: a case study on the spread and victims of an internet worm , 2002, IMW '02.

[16]  Kristopher Kendall,et al.  A Database of Computer Attacks for the Evaluation of Intrusion Detection Systems , 1999 .

[17]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[18]  David J. Marchette,et al.  Computer Intrusion Detection and Network Monitoring , 2001, Statistics for Engineering and Information Science.

[19]  Stephanie Forrest,et al.  Infect Recognize Destroy , 1996 .

[20]  Vern Paxson,et al.  Enhancing byte-level network intrusion detection signatures with context , 2003, CCS '03.

[21]  Robert Morris,et al.  Designing a framework for active worm detection on global networks , 2003, First IEEE International Workshop on Information Assurance, 2003. IWIAS 2003. Proceedings..

[22]  Angelos D. Keromytis,et al.  Using graphic turing tests to counter automated DDoS attacks against web servers , 2003, CCS '03.

[23]  Stuart Staniford,et al.  Towards Faster String Matching for Intrusion Detection , 2001 .

[24]  Christopher Krügel,et al.  Distributed Pattern Detection for Intrusion Detection , 2002, NDSS.

[25]  Michael I. Jordan,et al.  Computer Intrusion Detection and Network Monitoring: A Statistical Viewpoint , 2001 .

[26]  Michael Schatz,et al.  Learning Program Behavior Profiles for Intrusion Detection , 1999, Workshop on Intrusion Detection and Network Monitoring.

[27]  Yu Lin,et al.  Application intrusion detection using language library calls , 2001, Seventeenth Annual Computer Security Applications Conference.

[28]  John McHugh,et al.  Locality: a new paradigm for thinking about normal behavior and outsider threat , 2003, NSPW '03.

[29]  Jim Alves-Foss,et al.  An empirical analysis of NATE: Network Analysis of Anomalous Traffic Events , 2002, NSPW '02.

[30]  Vern Paxson,et al.  An analysis of using reflectors for distributed denial-of-service attacks , 2001, CCRV.

[31]  Christopher Kruegel,et al.  Connection-History Based Anomaly Detection , 2002 .

[32]  Yin Zhang,et al.  Detecting Backdoors , 2000, USENIX Security Symposium.

[33]  Jaideep Srivastava,et al.  A Comparative Study of Anomaly Detection Schemes in Network Intrusion Detection , 2003, SDM.

[34]  Dan Schnackenberg,et al.  Statistical approaches to DDoS attack detection and response , 2003, Proceedings DARPA Information Survivability Conference and Exposition.

[35]  Kyoji Kawagoe,et al.  A similarity search method of time series data with combination of Fourier and wavelet transforms , 2002, Proceedings Ninth International Symposium on Temporal Representation and Reasoning.

[36]  Sara Matzner,et al.  An application of machine learning to network intrusion detection , 1999, Proceedings 15th Annual Computer Security Applications Conference (ACSAC'99).

[37]  Sushil Jajodia,et al.  ADAM: Detecting Intrusions by Data Mining , 2001 .

[38]  Vern Paxson,et al.  How to Own the Internet in Your Spare Time , 2002, USENIX Security Symposium.

[40]  George C. Polyzos,et al.  A Parameterizable Methodology for Internet Traffic Flow Profiling , 1995, IEEE J. Sel. Areas Commun..

[41]  Angelos D. Keromytis,et al.  SOS: secure overlay services , 2002, SIGCOMM '02.

[42]  Kofi Nyarko,et al.  Network intrusion visualization with NIVA, an intrusion detection visual analyzer with haptic integration , 2002, Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002.

[43]  Calvin Ko,et al.  Logic induction of valid behavior specifications for intrusion detection , 2000, Proceeding 2000 IEEE Symposium on Security and Privacy. S&P 2000.

[44]  Kang G. Shin,et al.  Detecting SYN flooding attacks , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[45]  Jonathan Lemon,et al.  Resisting SYN Flood DoS Attacks with a SYN Cache , 2002, BSDCon.

[46]  Vern Paxson,et al.  Bro: a system for detecting network intruders in real-time , 1998, Comput. Networks.

[47]  Kevin A. Kwiat,et al.  Modeling the spread of active worms , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[48]  Salvatore J. Stolfo,et al.  Mining in a data-flow environment: experience in network intrusion detection , 1999, KDD '99.

[49]  John McHugh,et al.  Testing Intrusion detection systems: a critique of the 1998 and 1999 DARPA intrusion detection system evaluations as performed by Lincoln Laboratory , 2000, TSEC.

[50]  William L. Fithen,et al.  State of the Practice of Intrusion Detection Technologies , 2000 .

[51]  Isij Monitor,et al.  Network Intrusion Detection: An Analyst’s Handbook , 2000 .

[52]  Kai Hwang,et al.  Anomaly Intrusion Detection by Internet Datamining of Traffic Episodes , 2004 .

[53]  Angelos D. Keromytis,et al.  A network worm vaccine architecture , 2003, WET ICE 2003. Proceedings. Twelfth IEEE International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises, 2003..

[54]  Markus G. Kuhn,et al.  Analysis of a denial of service attack on TCP , 1997, Proceedings. 1997 IEEE Symposium on Security and Privacy (Cat. No.97CB36097).

[55]  Angelos D. Keromytis,et al.  Countering code-injection attacks with instruction-set randomization , 2003, CCS '03.

[56]  Marc Dacier,et al.  A Lightweight Tool for Detecting Web Server Attacks , 2000, NDSS.

[57]  Giovanni Vigna,et al.  NetSTAT: a network-based intrusion detection approach , 1998, Proceedings 14th Annual Computer Security Applications Conference (Cat. No.98EX217).

[58]  Peter Rodgers,et al.  Spring embedder preprocessing for WWW visualization , 2002, Proceedings Sixth International Conference on Information Visualisation.

[59]  Benno Stein,et al.  Visualization of traffic structures , 2001, ICC 2001. IEEE International Conference on Communications. Conference Record (Cat. No.01CH37240).

[60]  Peter Eades,et al.  A Heuristic for Graph Drawing , 1984 .

[61]  Cristina L. Abad,et al.  Log correlation for intrusion detection: a proof of concept , 2003, 19th Annual Computer Security Applications Conference, 2003. Proceedings..

[62]  Farnam Jahanian,et al.  Defeating TCP/IP Stack Fingerprinting , 2000, USENIX Security Symposium.

[63]  Sushil Jajodia,et al.  ADAM: a testbed for exploring the use of data mining in intrusion detection , 2001, SGMD.

[64]  R. Sekar,et al.  A fast automaton-based method for detecting anomalous program behaviors , 2001, Proceedings 2001 IEEE Symposium on Security and Privacy. S&P 2001.

[65]  Thomer M. Gil,et al.  MULTOPS: A Data-Structure for Bandwidth Attack Detection , 2001, USENIX Security Symposium.

[66]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[67]  Anup K. Ghosh,et al.  A Study in Using Neural Networks for Anomaly and Misuse Detection , 1999, USENIX Security Symposium.

[68]  Donald F. Towsley,et al.  Code red worm propagation modeling and analysis , 2002, CCS '02.

[69]  Chengqi Zhang,et al.  MA-IDS Architecture for Distributed Intrusion Detection using Mobile Agent , 2004 .

[70]  Christopher Krügel,et al.  Decentralized Event Correlation for Intrusion Detection , 2001, ICISC.

[71]  Matthew M. Williamson,et al.  Throttling viruses: restricting propagation to defeat malicious mobile code , 2002, 18th Annual Computer Security Applications Conference, 2002. Proceedings..

[72]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[73]  Wei Jiang,et al.  The Mahalanobis–Taguchi Strategy , 2003, Technometrics.

[74]  Richard A. Kemmerer,et al.  State Transition Analysis: A Rule-Based Intrusion Detection Approach , 1995, IEEE Trans. Software Eng..

[75]  Risto Miikkulainen,et al.  Intrusion Detection with Neural Networks , 1997, NIPS.

[76]  Kenneth M. Hall.  An r-Dimensional Quadratic Placement Algorithm , 1970 .

[77]  Krishna Bharat,et al.  Who links to whom: mining linkage between Web sites , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[78]  Mian Zhou,et al.  Mining Frequency Content of Network Traffic for Intrusion Detection , 2003 .

[79]  H. Javitz,et al.  Detecting Unusual Program Behavior Using the Statistical Component of the Next-generation Intrusion Detection Expert System ( NIDES ) 1 , 1997 .

[80]  Stephanie Forrest,et al.  An immunological model of distributed detection and its application to computer security , 1999 .

[81]  Martin Roesch,et al.  Snort - Lightweight Intrusion Detection for Networks , 1999 .

[82]  Susan M. Bridges,et al.  FUZZY DATA MINING AND GENETIC ALGORITHMS APPLIED TO INTRUSION DETECTION , 2002 .

[83]  Sheau-Dong Lang,et al.  Weighted link graphs: a distributed IDS for secondary intrusion detection and defense , 2005, SPIE Defense + Commercial Sensing.