A critical evaluation of datasets for investigating IDSs and IPSs researches

Complex and new cases of intrusions, new bugs, security issues and vulnerabilities are evolving everyday for a number of reasons. Consequently, researchers in the domains of Intrusion Detection Systems and Intrusion Prevention Systems constantly design new methods to lessen the aforementioned security issues. However, getting suitable datasets for evaluating various research designs in these domains is a major challenge for the research community, vendors and data donors over the years. As a result, most intrusion detection and prevention methodologies are evaluated using wrong categories of datasets because the limitations of each category of evaluative datasets are unknown. Therefore, this paper presents a critique of the challenges associated with evaluative datasets for investigating intrusion detection and prevention methodologies and how these challenges can be lessened. Finally, these analyses will effective guide researchers and vendors in securing evaluative datasets for validating the intrusion detection and prevention systems.

[1]  W. Timothy Strayer,et al.  SLINGbot: A System for Live Investigation of Next Generation Botnets , 2009, 2009 Cybersecurity Applications & Technology Conference for Homeland Security.

[2]  J. Chow.  An Assessment of the DARPA IDS Evaluation Dataset Using Snort S Terry Brugger , 2005 .

[3]  Basil Abdullah AsSadhan,et al.  Network traffic analysis through statistical signal processing methods , 2009 .

[4]  Shilpa Lakhina,et al.  Feature Reduction using Principal Component Analysis for Effective Anomaly – Based Intrusion Detection on NSL-KDD , 2010 .

[5]  Sally Floyd,et al.  Why we don't know how to simulate the Internet , 1997, WSC '97.

[6]  Philip K. Chan,et al.  An Analysis of the 1999 DARPA/Lincoln Laboratory Evaluation Data for Network Anomaly Detection , 2003, RAID.

[7]  Peter Mell,et al.  NIST Special Publication on Intrusion Detection Systems , 2001 .

[8]  John Heidemann,et al.  Uses and Challenges for Network Datasets , 2009, 2009 Cybersecurity Applications & Technology Conference for Homeland Security.

[9]  Ali A. Ghorbani,et al.  Network Intrusion Detection and Prevention - Concepts and Techniques , 2010, Advances in Information Security.

[10]  Vishwas Sharma,et al.  Usefulness of DARPA dataset for intrusion detection system evaluation , 2008, SPIE Defense + Commercial Sensing.

[11]  Paul D. Scott,et al.  Evaluating data mining procedures: techniques for generating artificial data sets , 1999, Inf. Softw. Technol..

[12]  Richard Macfarlane,et al.  A methodology to evaluate rate-based intrusion prevention system against distributed denial-of-service (DDoS). , 2011 .

[13]  John McHugh,et al.  Testing Intrusion detection systems: a critique of the 1998 and 1999 DARPA intrusion detection system evaluations as performed by Lincoln Laboratory , 2000, TSEC.