Remote Profiling of Resource Constraints of Web Servers Using Mini-Flash Crowds

Unexpected surges in Web request traffic can exercise server-side resources (e.g., access bandwidth, processing, storage etc.) in undesirable ways. Administrators today do not have requisite tools to understand the impact of such "flash crowds" on their servers. Most Web servers either rely on over-provisioning and admission control, or use potentially expensive solutions like CDNs, to ensure high availability in the face of flash crowds. A more fine-grained understanding of the performance of individual server resources under emulated but realistic and controlled flash crowd-like conditions can aid administrators to make more efficient resource management decisions. In this paper, we present miniflash crowds (MFC) - a light-weight profiling service that reveals resource bottlenecks in a Web server infrastructure. MFC uses a set of controlled probes where an increasing number of distributed clients make synchronized requests that exercise specific resources or portions of a remote Web server. We carried out controlled lab-based tests and experiments in collaboration with operators of production servers. We show that our approach can faithfully track the impact of request loads on different server resources and provide useful insights to server operators on the constraints of different components of their infrastructure. We also present results from a measurement study of the provisioning of several hundred popular Web servers, a few hundred Web servers of startup companies, and about hundred phishing servers.

[1]  Jelena Mirkovic,et al.  A realistic simulation of internet-scale events , 2006, valuetools '06.

[2]  Elaine Shi,et al.  Portcullis: protecting connection setup from denial-of-capability attacks , 2007, SIGCOMM '07.

[3]  Xiaowei Yang,et al.  A DoS-limiting network architecture , 2005, SIGCOMM '05.

[4]  Sally Floyd,et al.  Identifying the tcp behavior of web servers , 2000, SIGCOMM 2000.

[5]  Martín Casado,et al.  Cookies Along Trust-Boundaries (CAT): Accurate and Deployable Flood Protection , 2006, SRUTI.

[6]  Richard Mortier,et al.  Using Magpie for Request Extraction and Workload Modelling , 2004, OSDI.

[7]  Michael Walfish,et al.  DDoS defense by offense , 2006, SIGCOMM 2006.

[8]  Yin Zhang,et al.  On the constancy of internet path properties , 2001, IMW '01.

[9]  Marcos K. Aguilera,et al.  Performance debugging for distributed systems of black boxes , 2003, SOSP '03.

[10]  John S. Heidemann,et al.  Flash crowd mitigation via adaptive admission control based on application-level observations , 2005, TOIT.

[11]  John S. Sobolewski,et al.  SideCar: building programmable datacenter networks without programmable switches , 2010, Hotnets-IX.

[12]  Michael Walfish,et al.  DDoS defense by offense , 2006, TOCS.

[13]  Vitaly Shmatikov,et al.  dFence: Transparent Network-based Denial of Service Mitigation , 2007, NSDI.

[14]  David Mosberger,et al.  httperf—a tool for measuring web server performance , 1998, PERV.

[15]  Bo Hong,et al.  Managing flash crowds on the Internet , 2003, 11th IEEE/ACM International Symposium on Modeling, Analysis and Simulation of Computer Telecommunications Systems, 2003. MASCOTS 2003..

[16]  Balachander Krishnamurthy,et al.  Flash crowds and denial of service attacks: characterization and implications for CDNs and web sites , 2002, WWW.

[17]  Elaine Shi,et al.  Portcullis: protecting connection setup from denial-of-capability attacks , 2007, SIGCOMM 2007.

[18]  David Mosberger,et al.  A tool for measuring web server performance , 1998, SIGMETRICS 1998.