A Framework of Evaluation Methodologies for Network Anomaly Detectors

Anomaly detection has been a field of intensive research over the last years. Along with that several works to evaluate anomaly detectors have been proposed. In this paper we argue four properties regarding ideal evaluation methodologies that cannot be answered by single current evaluation technique employed today. We therefore present an framework of an evaluation methodology that leverages traces from operational networks, simulation and emulation to satisfy the four properties.

[1]  Dongho Kim,et al.  Experience with DETER: a testbed for security research , 2006, 2nd International Conference on Testbeds and Research Infrastructures for the Development of Networks and Communities, 2006. TRIDENTCOM 2006..

[2]  Steve Uhlig,et al.  Providing public intradomain traffic matrices to the research community , 2006, CCRV.

[3]  George Varghese,et al.  Automatically inferring patterns of resource consumption in network traffic , 2003, SIGCOMM '03.

[4]  Paul Barford,et al.  Self-configuring network traffic generation , 2004, IMC '04.

[5]  Vinod Yegneswaran,et al.  A framework for malicious workload generation , 2004, IMC '04.

[6]  Nick Feamster,et al.  Diagnosing network disruptions with network-wide analysis , 2007, SIGMETRICS '07.

[7]  Eddie Kohler,et al.  Internet research needs better models , 2003, CCRV.

[8]  Catherine Rosenberg,et al.  Evaluation methods for internet security technology , 2004 .

[9]  Ramesh Govindan,et al.  Detection and identification of network anomalies using sketch subspaces , 2006, IMC '06.

[10]  Shang Wen-zhong,et al.  ODC: a method for online detecting & classifying network-wide traffic anomalies , 2011 .

[11]  Matthew Roughan,et al.  The need for simulation in evaluating anomaly detectors , 2008, CCRV.

[12]  Jennifer Rexford,et al.  WebClass: adding rigor to manual labeling of traffic anomalies , 2008, CCRV.

[13]  Christophe Diot,et al.  Diagnosing network-wide traffic anomalies , 2004, SIGCOMM.

[14]  Sonia Fahmy,et al.  DDoS Benchmarks and Experimenter's Workbench for the DETER Testbed , 2007, 2007 3rd International Conference on Testbeds and Research Infrastructure for the Development of Networks and Communities.

[15]  Paul Barford,et al.  Generating representative Web workloads for network and server performance evaluation , 1998, SIGMETRICS '98/PERFORMANCE '98.