A monitoring system for detecting repeated packets with applications to computer worms

We present a monitoring system which detects repeated packets in network traffic, and has applications including detecting computer worms. It uses Bloom filters with counters. The system analyzes traffic in routers of a network. Our preliminary evaluation of the system involved traffic from our internal lab and a well known historical data set. After appropriate configuration, no false alarms are obtained under these data sets and we expect low false alarm rates are possible in many network environments. We also conduct simulations using real Internet Service Provider topologies with realistic link delays and simulated traffic. These simulations confirm that this approach can detect worms at early stages of propagation. We believe our approach, with minor adaptations, is of independent interest for use in a number of network applications which benefit from detecting repeated packets, beyond detecting worm propagation. These include detecting network anomalies such as dangerous traffic fluctuations, abusive use of certain services, and some distributed denial-of-service attacks.

[1]  Matthew M. Williamson,et al.  Implementing and Testing a Virus Throttle , 2003, USENIX Security Symposium.

[2]  Andrei Broder,et al.  Network Applications of Bloom Filters: A Survey , 2004, Internet Math..

[3]  Eu-Jin Goh,et al.  Secure Indexes , 2003, IACR Cryptol. ePrint Arch..

[4]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1951 .

[5]  Stefan Savage,et al.  Inside the Slammer Worm , 2003, IEEE Secur. Priv..

[6]  David Moore,et al.  The Spread of the Witty Worm , 2004, IEEE Secur. Priv..

[7]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[8]  Hari Balakrishnan,et al.  Fast portscan detection using sequential hypothesis testing , 2004, IEEE Symposium on Security and Privacy, 2004. Proceedings. 2004.

[9]  Paul C. van Oorschot,et al.  Mitigating Network Denial-of-Service Through Diversity-Based Traffic Management , 2005, ACNS.

[10]  Andrei Z. Broder,et al.  Graph structure in the Web , 2000, Comput. Networks.

[11]  K. Joag-dev,et al.  Negative Association of Random Variables with Applications , 1983 .

[12]  Burton H. Bloom,et al.  Space/time trade-offs in hash coding with allowable errors , 1970, CACM.

[13]  M. Koutras,et al.  On the number of overflown urns and excess balls in an allocation model with limited urn capacity , 2002 .

[14]  Matthew M. Williamson,et al.  Throttling viruses: restricting propagation to defeat malicious mobile code , 2002, 18th Annual Computer Security Applications Conference, 2002. Proceedings..

[15]  James Newsome,et al.  Polygraph: automatically generating signatures for polymorphic worms , 2005, 2005 IEEE Symposium on Security and Privacy (S&P'05).

[16]  Craig Partridge,et al.  Hash-based IP traceback , 2001, SIGCOMM.

[17]  Dimitri P. Bertsekas,et al.  Data Networks , 1986 .

[18]  Andrew Smith,et al.  Digging for worms, fishing for answers , 2002, 18th Annual Computer Security Applications Conference, 2002. Proceedings..

[19]  Robert K. Cunningham,et al.  A taxonomy of computer worms , 2003, WORM '03.

[20]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1967 .

[21]  Christopher Kruegel,et al.  Connection-History Based Anomaly Detection , 2002 .

[22]  George Varghese,et al.  Automated Worm Fingerprinting , 2004, OSDI.

[23]  Heejo Lee,et al.  On the effectiveness of route-based packet filtering for distributed DoS attack prevention in power-law internets , 2001, SIGCOMM 2001.

[24]  Nasir D. Memon,et al.  Payload attribution via hierarchical bloom filters , 2004, CCS '04.

[25]  Heejo Lee,et al.  On the effectiveness of route-based packet filtering for distributed DoS attack prevention in power-law internets , 2001, SIGCOMM '01.

[26]  Thomas Henry Ptacek,et al.  Insertion, Evasion, and Denial of Service: Eluding Network Intrusion Detection , 1998 .

[27]  Alfonso Valdes,et al.  Scalable visualization of propagating internet phenomena , 2004, VizSEC/DMSEC '04.

[28]  Michalis Faloutsos,et al.  On power-law relationships of the Internet topology , 1999, SIGCOMM '99.

[29]  Elias Levy.  Worm Propagation and Generic Attacks , 2005, IEEE Secur. Priv..

[30]  John Heidemann,et al.  Detecting Early Worm Propagation through Packet Matching , 2004 .

[31]  Ratul Mahajan,et al.  Measuring ISP topologies with rocketfuel , 2002, TNET.

[32]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[33]  Sarang Dharmapurikar,et al.  Robust TCP Stream Reassembly in the Presence of Adversaries , 2005, USENIX Security Symposium.

[34]  Thomas H. Cormen,et al.  Introduction to algorithms [2nd ed.] , 2001 .

[35]  Vern Paxson,et al.  Proceedings of the 13th USENIX Security Symposium , 2022 .

[36]  B. Karp,et al.  Autograph: Toward Automated, Distributed Worm Signature Detection , 2004, USENIX Security Symposium.

[37]  David E. Taylor,et al.  Longest prefix matching using bloom filters , 2006, TNET.

[38]  Sumeet Singh,et al.  The EarlyBird System for Real-time Detection of Unknown Worms , 2005 .

[39]  Li Fan,et al.  Summary cache: a scalable wide-area web cache sharing protocol , 2000, TNET.

[40]  D. Watts,et al.  Small Worlds: The Dynamics of Networks between Order and Randomness , 2001 .

[41]  Donald F. Towsley,et al.  Code red worm propagation modeling and analysis , 2002, CCS '02.

[42]  Dan S. Wallach,et al.  Denial of Service via Algorithmic Complexity Attacks , 2003, USENIX Security Symposium.

[43]  A. Barbour,et al.  Poisson Approximation , 1992 .

[44]  W. Feller,et al.  An Introduction to Probability Theory and Its Applications, Vol. 1 , 1967 .

[45]  Miguel Vargas Martin.  AM ONITORING SYSTEM FOR MITIGATING FAST PROPAGATING WORMS IN THE NETWORK INFRASTRUCTURE , 2005 .

[46]  John W. Lockwood,et al.  Deep packet inspection using parallel bloom filters , 2004, IEEE Micro.

[47]  Philip K. Chan,et al.  An Analysis of the 1999 DARPA/Lincoln Laboratory Evaluation Data for Network Anomaly Detection , 2003, RAID.

[48]  Robert E. Tarjan,et al.  Dynamic Self-Checking Techniques for Improved Tamper Resistance , 2001, Digital Rights Management Workshop.

[49]  Matthew M. Williamson.  Design, implementation and test of an email virus throttle , 2003, 19th Annual Computer Security Applications Conference, 2003. Proceedings..

[50]  Dawn Xiaodong Song,et al.  New Streaming Algorithms for Fast Detection of Superspreaders , 2005, NDSS.

[51]  Salvatore J. Stolfo,et al.  Anomalous Payload-Based Network Intrusion Detection , 2004, RAID.

[52]  Carey Nachenberg,et al.  Computer virus-antivirus coevolution , 1997, Commun. ACM.

[53]  Ratul Mahajan,et al.  Inferring link weights using end-to-end measurements , 2002, IMW '02.

[54]  John McHugh,et al.  Testing Intrusion detection systems: a critique of the 1998 and 1999 DARPA intrusion detection system evaluations as performed by Lincoln Laboratory , 2000, TSEC.

[55]  Ali A. Ghorbani,et al.  A novel visualization technique for network anomaly detection , 2004, PST.

[56]  Daryl J. Daley,et al.  Epidemic Modelling: An Introduction , 1999 .

[57]  Abhishek Kumar,et al.  Space-code bloom filter for efficient traffic flow measurement , 2003, IMC '03.

[58]  Mark Handley,et al.  Network Intrusion Detection: Evasion, Traffic Normalization, and End-to-End Protocol Semantics , 2001, USENIX Security Symposium.

[59]  Bart Preneel,et al.  Software Performance of Universal Hash Functions , 1999, EUROCRYPT.

[60]  Nghi Nguyen,et al.  Comparative Analysis of the Hardware Implementations of Hash Functions SHA-1 and SHA-512 , 2002, ISC.

[61]  Michael Mitzenmacher,et al.  Compressed bloom filters , 2001, PODC '01.

[62]  Charles M. Grinstead,et al.  Introduction to probability , 1999, Statistics for the Behavioural Sciences.