Distributed online flash-crowd detection in P2P swarming systems

Peer-to-peer applications generate most of the Internet traffic and have become an important determining factor for upgrading Internet backbone capacity. It is thus important to assure that these systems attain high performance and deliver good quality of service to their users. Thus, apart from off-line analysis of traces, online mechanisms for estimating real-time changes of the network characteristics (i.e., network size, churn, failures, etc.) are needed to enable the design of adaptive algorithms. In this paper we focus on the problem of online detection of the flash-crowd phenomenon, defined as a sudden, unexpected increase in the number of peers requesting a piece of content. The main contribution of the paper is made out of two distributed algorithms that allow peers to detect flash-crowds, using a gossiping alike protocol. To the best of our knowledge, this is the first online detection method for the flash-crowd phenomenon. We base our algorithms on a modified version of gossiping, where peers are requested to periodically, asynchronously, reset their local mass variable. By using different reset values for the joining nodes, the algorithms create distributed network aggregates that reflect the sudden relative increase in network size (as part of FlashDetect algorithm) or the absolute network size (as part of TrackerNetSize algorithm). We analyze the performance of the proposed algorithms and perform extensive simulations to showcase their behavior on both synthetic data as well as real-world traces. The results show that our proposed solution performs very well in both cases, achieving high detection rates of the flash-crowd phenomenon within short time intervals while keeping the traffic load at a minimum. Additionally, the comparison with related work shows that TrackerNetSize achieves similar results with current state-of-the-art network size estimation algorithms, while making use of a significantly reduced assumption set (in particular, the peers do not have to be synchronized and, tracking the departing peers is not required).