Energy-price-driven query processing in multi-center web search engines

Concurrently processing thousands of web queries, each with a response time under a fraction of a second, necessitates maintaining and operating massive data centers. For large-scale web search engines, this translates into high energy consumption and a huge electric bill. This work takes the challenge to reduce the electric bill of commercial web search engines operating on data centers that are geographically far apart. Based on the observation that energy prices and query workloads show high spatio-temporal variation, we propose a technique that dynamically shifts the query workload of a search engine between its data centers to reduce the electric bill. Experiments on real-life query workloads obtained from a commercial search engine show that significant financial savings can be achieved by this technique.

[1]  Abdur Chowdhury,et al.  Operational requirements for scalable search systems , 2003, CIKM '03.

[2]  Philip S. Yu,et al.  Dynamic Load Balancing on Web-Server Systems , 1999, IEEE Internet Comput..

[3]  Thomas F. Wenisch,et al.  PowerNap: eliminating server idle power , 2009, ASPLOS.

[4]  A.J. Shah,et al.  Optimization of Global Data Center Thermal Management Workload for Minimal Environmental and Economic Burden , 2008, IEEE Transactions on Components and Packaging Technologies.

[5]  Aristides Gionis,et al.  On the feasibility of multi-site web search engines , 2009, CIKM.

[6]  Vivek S. Pai,et al.  The effectiveness of request redirection on CDN robustness , 2002, OSDI '02.

[7]  Xue Liu,et al.  Minimizing Electricity Cost: Optimization of Distributed Internet Data Centers in a Multi-Electricity-Market Environment , 2010, 2010 Proceedings IEEE INFOCOM.

[8]  Luiz André Barroso,et al.  The Price of Performance , 2005, ACM Queue.

[9]  Margaret Martonosi,et al.  Capping the brown energy consumption of Internet services at low cost , 2010, International Conference on Green Computing.

[10]  Niraj Tolia,et al.  Opportunities and challenges to unify workload, power, and cooling management in data centers , 2010, OPSR.

[11]  Luiz André Barroso,et al.  The Case for Energy-Proportional Computing , 2007, Computer.

[12]  Berkant Barla Cambazoglu,et al.  Query forwarding in geographically distributed search engines , 2010, SIGIR.

[13]  Craig MacDonald,et al.  Terrier Information Retrieval Platform , 2005, ECIR.

[14]  Berkant Barla Cambazoglu,et al.  A refreshing perspective of search engine caching , 2010, WWW '10.

[15]  Bruce M. Maggs,et al.  Cutting the electric bill for internet-scale systems , 2009, SIGCOMM '09.

[16]  Xue Liu,et al.  MEC-IDC: joint load balancing and power control for distributed Internet Data Centers , 2010, ICCPS '10.

[17]  Luiz André Barroso,et al.  Web Search for a Planet: The Google Cluster Architecture , 2003, IEEE Micro.

[18]  Supranamaya Ranjan,et al.  Wide area redirection of dynamic content by Internet data centers , 2004, IEEE INFOCOM 2004.

[19]  Luiz André Barroso,et al.  The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines , 2009, The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines.

[20]  Berkant Barla Cambazoglu,et al.  Quantifying performance and quality gains in distributed web search engines , 2009, SIGIR.

[21]  Torsten Suel,et al.  Improved techniques for result caching in web search engines , 2009, WWW '09.