Automatic Evaluation of Intrusion Detection Systems

An intrusion detection system (IDS) is a crucial element of a network security posture. Although there are many IDS products available, it is rather difficult to find information about their accuracy. Only a few organizations evaluate these products. Furthermore, the data used to test and evaluate these IDS is usually proprietary. Thus, the research community cannot easily evaluate the next generation of IDS. Toward this end, DARPA provided in 1998, 1999 and 2000 an intrusion detection evaluation data set. However, no new data set has been released by DARPA since 2000, in part because of the cumbersomeness of the task. In this paper, we propose a strategy to address certain aspects of generating a publicly available documented data set for testing and evaluating intrusion detection systems. We also present a tool that automatically analyzes and evaluates IDS using our proposed data set

[1]  Annie De Montigny-Leboeuf.  A Multi-Packet Signature Approach to Passive Operating System Detection , 2005 .

[2]  Helen J. Wang,et al.  Virtual Playgrounds for Worm Behavior Investigation , 2005, RAID.

[3]  Thomas Henry Ptacek,et al.  Insertion, Evasion, and Denial of Service: Eluding Network Intrusion Detection , 1998 .

[4]  Greg Shipley,et al.  Cover story: dragon claws its way to the top , 2001 .

[5]  Richard P. Lippmann,et al.  An Overview of Issues in Testing Intrusion Detection Systems , 2003 .

[6]  D. J. Roelker.  HTTP IDS Evasions Revisited , 2004 .

[7]  John McHugh,et al.  Testing Intrusion detection systems: a critique of the 1998 and 1999 DARPA intrusion detection system evaluations as performed by Lincoln Laboratory , 2000, TSEC.

[8]  Giovanni Vigna,et al.  An experience developing an IDS stimulator for the black-box testing of network intrusion detection systems , 2003, 19th Annual Computer Security Applications Conference, 2003. Proceedings..

[9]  Marc Dacier,et al.  ScriptGen: an automated script generation tool for Honeyd , 2005, 21st Annual Computer Security Applications Conference (ACSAC'05).

[10]  Calcagnini Giovanni.  Fun with Packets: Designing a Stick , 2002 .

[11]  Hervé Debar,et al.  Evaluation of the Diagnostic Capabilities of Commercial Intrusion Detection Systems , 2002, RAID.

[12]  Vern Paxson,et al.  Bro: a system for detecting network intruders in real-time , 1998, Comput. Networks.

[13]  Jesse C. Rabek,et al.  LARIAT: Lincoln adaptable real-time information assurance testbed , 2002, Proceedings, IEEE Aerospace Conference.

[14]  Giovanni Vigna,et al.  Testing network-based intrusion detection signatures using mutant exploits , 2004, CCS '04.

[15]  Michael Vrable,et al.  Scalability, fidelity, and containment in the potemkin virtual honeyfarm , 2005, SOSP '05.

[16]  George F. Riley,et al.  Intrusion detection testing and benchmarking methodologies , 2003, First IEEE International Workshop on Information Assurance, 2003. IWIAS 2003. Proceedings..

[17]  Jeffrey Posluns,et al.  Snort 2.0 Intrusion Detection , 2003 .

[18]  Jeff Dike,et al.  User-mode Linux , 2006, Annual Linux Showcase & Conference.

[19]  Mark Handley,et al.  Network Intrusion Detection: Evasion, Traffic Normalization, and End-to-End Protocol Semantics , 2001, USENIX Security Symposium.