Bridging the Data Gap: Data Related Challenges in Evaluating Large Scale Collaborative Security Systems

Data-sharing approaches such as collaborative security have been successfully applied to systems addressing multiple classes of cyber security threats. In spite of these results, scale presents a major challenge to further advances: collaborative security systems are designed to operate at a large scale (Internetor ISP-scale), and obtaining and sharing traces suitable for experimentation is difficult. We illustrate these challenges via an analysis of recently proposed collaborative systems. We argue for the development of simulation techniques designed specifically to address these challenges and sketch one such technique, parameterized trace scaling, which expands small traces to generate realistic large scale traces sufficient for analyzing collaborative security systems.

[1]  David Moore,et al.  Internet quarantine: requirements for containing self-propagating code , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[2]  Guofei Gu,et al.  BotMiner: Clustering Analysis of Network Traffic for Protocol- and Structure-Independent Botnet Detection , 2008, USENIX Security Symposium.

[3]  Charles V. Wright,et al.  Playing Devil's Advocate: Inferring Sensitive Information from Anonymized Network Traces , 2007, NDSS.

[4]  Thorsten Holz,et al.  Rishi: Identify Bot Contaminated Hosts by IRC Nickname Evaluation , 2007, HotBots.

[5]  Phillip A. Porras,et al.  Highly Predictive Blacklisting , 2008, USENIX Security Symposium.

[6]  Somesh Jha,et al.  Global Intrusion Detection in the DOMINO Overlay System , 2004, NDSS.

[7]  Felix C. Freiling,et al.  The Nepenthes Platform: An Efficient Approach to Collect Malware , 2006, RAID.

[8]  Herbert Bos,et al.  Prudent Practices for Designing Malware Experiments: Status Quo and Outlook , 2012, 2012 IEEE Symposium on Security and Privacy.

[9]  Matthew Roughan,et al.  The need for simulation in evaluating anomaly detectors , 2008, CCRV.

[10]  JahanianFarnam,et al.  The Blaster Worm , 2005, S&P 2005.

[11]  Bernhard Plattner,et al.  Entropy based worm and anomaly detection in fast IP networks , 2005, 14th IEEE International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprise (WETICE'05).

[12]  Michele C. Weigle,et al.  Tmix: a tool for generating realistic TCP application workloads in ns-2 , 2006, CCRV.

[13]  Nick McKeown,et al.  A network in a laptop: rapid prototyping for software-defined networks , 2010, Hotnets-IX.

[14]  Andreas Haeberlen,et al.  Challenges in Experimenting with Botnet Detection Systems , 2011, CSET.

[15]  Jason Lee,et al.  The devil and packet trace anonymization , 2006, CCRV.

[16]  Vern Paxson,et al.  Bro: a system for detecting network intruders in real-time , 1998, Comput. Networks.

[17]  George F. Riley,et al.  The Georgia Tech Network Simulator , 2003, MoMeTools '03.

[18]  Balachander Krishnamurthy,et al.  Collaborating against common enemies , 2005, IMC '05.

[19]  B. Karp,et al.  Autograph: Toward Automated, Distributed Worm Signature Detection , 2004, USENIX Security Symposium.

[20]  Dirk Grunwald,et al.  Legal issues surrounding monitoring during network research , 2007, IMC '07.

[21]  David Moore,et al.  The Spread of the Witty Worm , 2004, IEEE Secur. Priv..

[22]  Klaus Wehrle,et al.  Proceedings of the ACM SIGCOMM workshop on Models, methods and tools for reproducible network research , 2003, SOSP 2003.

[23]  Salvatore J. Stolfo,et al.  Cross-Domain Collaborative Anomaly Detection: So Far Yet So Close , 2011, RAID.

[24]  Prateek Mittal,et al.  BotGrep: Finding P2P Bots with Structured Graph Analysis , 2010, USENIX Security Symposium.

[25]  David Watson,et al.  The Blaster worm: then and now , 2005, IEEE Security & Privacy Magazine.