Cyber Disease Monitoring with Distributed Hash Tables : A Global Peer-to-Peer Intrusion Detection System

Traffic anomalies and distributed attacks are commonplace in today’s networks. Single point detection is often insufficient to determine the causes, patterns and prevalence of such events. Most existing distributed intrusion detection systems (DIDS) rely on centralized fusion, or distributed fusion with unscalable communication mechanisms. In this paper, we propose to build a distributed IDS based on the emerging decentralized location and routing infrastructure: distributed hash table (DHT). We embed the intrusion symptoms into the DHT dimensions so that alarms related to the same intrusion (thus with similar symptoms) will be routed to the same sensor fusion center (SFC) while evenly distributing unrelated alarms to different SFCs. This is achieved through careful routing key design based on: 1) analysis of essential characteristics of three common types of intrusions: DoS attacks, port scanning and virus/worm infection; and 2) distribution and stability analysis of the popular port numbers and those of the popular source IP addresses in scans. We further propose load-aware node bootstrapping to distribute the alarms more evenly across the fusion centers. Evaluation based on one month of DShield firewall logs (600 million scan records) collected from over 2200 worldwide providers show that the resulting system, termed Cyber Disease DHT (CDDHT), can effectively fuse related alarms while distributing unrelated ones evenly among the SFCs. Open questions on querying and attack-resilience of CDDHT are also discussed.

[1]  Karl N. Levitt,et al.  GrIDS A Graph-Based Intrusion Detection System for Large Networks , 1996 .

[2]  Stefan Axelsson,et al.  Intrusion Detection Systems: A Survey and Taxonomy , 2002 .

[3]  Robert K. Cunningham,et al.  Large Scale Malicious Code: A Research Agenda , 2003 .

[4]  Hari Balakrishnan,et al.  Fast portscan detection using sequential hypothesis testing , 2004, IEEE Symposium on Security and Privacy, 2004. Proceedings. 2004.

[5]  Sandhya Dwarkadas,et al.  Peer-to-peer information retrieval using self-organizing semantic overlay networks , 2003, SIGCOMM '03.

[6]  Vinod Yegneswaran,et al.  Internet intrusions: global characteristics and prevalence , 2003, SIGMETRICS '03.

[7]  Tim Bass,et al.  Intrusion detection systems and multisensor data fusion , 2000, CACM.

[8]  Miguel Castro,et al.  SplitStream: High-Bandwidth Content Distribution in Cooperative Environments , 2003, IPTPS.

[9]  Alfonso Valdes,et al.  Probabilistic Alert Correlation , 2001, Recent Advances in Intrusion Detection.

[10]  Miguel Castro,et al.  SCRIBE: The Design of a Large-Scale Event Notification Infrastructure , 2001, Networked Group Communication.

[11]  Peter Druschel,et al.  Pastry: Scalable, distributed object location and routing for large-scale peer-to- , 2001 .

[12]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM '01.

[13]  Peter G. Neumann,et al.  EMERALD: Event Monitoring Enabling Responses to Anomalous Live Disturbances , 1997, CCS 2002.

[14]  Qi Zhang,et al.  Indra: a peer-to-peer approach to network intrusion detection and prevention , 2003, WET ICE 2003. Proceedings. Twelfth IEEE International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises, 2003..

[15]  Scott Shenker,et al.  Range Queries over DHTs , 2003 .

[16]  Michael Steiner,et al.  Malicious- and Accidental-Fault Tolerance for Internet Applications , 2002 .

[17]  William L. Fithen,et al.  State of the Practice of Intrusion Detection Technologies , 2000 .

[18]  Ben Y. Zhao,et al.  Tapestry: a resilient global-scale overlay for service deployment , 2004, IEEE Journal on Selected Areas in Communications.

[19]  Richard A. Kemmerer,et al.  NSTAT: A Model-based Real-time Network Intrusion Detection System , 1998 .

[20]  Stuart Staniford-Chen,et al.  Practical Automated Detection of Stealthy Portscans , 2002, J. Comput. Secur..

[21]  Peter Reiher,et al.  A taxonomy of DDoS attack and DDoS defense mechanisms , 2004, CCRV.

[22]  Balachander Krishnamurthy,et al.  Flash crowds and denial of service attacks: characterization and implications for CDNs and web sites , 2002, WWW.

[23]  Charles E. Kahn,et al.  A common intrusion detection framework , 2000 .

[24]  Christopher Krügel,et al.  Decentralized Event Correlation for Intrusion Detection , 2001, ICISC.

[25]  Stefan Savage,et al.  Inferring Internet denial-of-service activity , 2001, TOCS.

[26]  Salvatore J. Stolfo,et al.  Collaborative Distributed Intrusion Detection , 2004 .

[27]  Stefan Savage,et al.  The Spread of the Sapphire/Slammer Worm , 2003 .

[28]  Frédéric Cuppens,et al.  Alert correlation in a cooperative intrusion detection framework , 2002, Proceedings 2002 IEEE Symposium on Security and Privacy.

[29]  Sven Dietrich,et al.  Analyzing Distributed Denial of Service Tools: The Shaft Case , 2000, LISA.

[30]  Somesh Jha,et al.  Global Intrusion Detection in the DOMINO Overlay System , 2004, NDSS.

[31]  B. Cohen,et al.  Incentives Build Robustness in Bit-Torrent , 2003 .

[32]  Mark Handley,et al.  Application-Level Multicast Using Content-Addressable Networks , 2001, Networked Group Communication.

[33]  Eugene H. Spafford,et al.  An architecture for intrusion detection using autonomous agents , 1998, Proceedings 14th Annual Computer Security Applications Conference (Cat. No.98EX217).

[34]  Robert K. Cunningham,et al.  A taxonomy of computer worms , 2003, WORM '03.

[35]  Randy H. Katz,et al.  Quantifying Network Denial of Service: A Location Service Case Study , 2001, ICICS.

[36]  Biswanath Mukherjee,et al.  DIDS (distributed intrusion detection system)—motivation, architecture, and an early prototype , 1997 .

[37]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[38]  Vern Paxson,et al.  How to Own the Internet in Your Spare Time , 2002, USENIX Security Symposium.

[39]  David Moore,et al.  Internet quarantine: requirements for containing self-propagating code , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).