Counting distinct elements over sliding windows

In Distributed Denial of Service (DDoS) attacks, an attacker tries to disable a service with a flood of seemingly legitimate requests from multiple devices; this is usually accompanied by a sharp spike in the number of distinct IP addresses / flows accessing the system in a short time frame. Hence, the number of distinct elements over sliding windows is a fundamental signal in DDoS identification. Additionally, assessing whether a specific flow has recently accessed the system, known as the Set Membership problem, can help us identify the attacking parties. Here, we show how to extend the functionality of a state of the art algorithm for set membership over a W elements sliding window. We now also support estimation of the distinct flow count, using as little as log2 (W) additional bits.

[1]  Roy Friedman,et al.  Poster abstract: A sliding counting bloom filter , 2017, 2017 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[2]  Roy Friedman,et al.  Efficient Network Measurements through Approximated Windows , 2017, ArXiv.

[3]  Rajeev Rastogi,et al.  Streaming Algorithms for Robust, Real-Time Detection of DDoS Attacks , 2007, 27th International Conference on Distributed Computing Systems (ICDCS '07).

[4]  Moni Naor,et al.  Sliding Bloom Filters , 2013, ISAAC.

[5]  Frédéric Giroire,et al.  Estimating the Number of Active Flows in a Data Stream over a Sliding Window , 2007, ANALCO.

[6]  Roy Friedman,et al.  Counting with TinyTable: Every bit counts! , 2015, 2015 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).