Benchmarking Models and Tools for Distributed Web-Server Systems

This tutorial reviews benchmarking tools and techniques that can be used to evaluate the performance and scalability of highly accessed Web-server systems. The focus is on design and testing of locally and geographically distributed architectures where the performance evaluation is obtained through workload generators and analyzers in a laboratory environment. The tutorial identifies the qualities and issues of existing tools with respect to the main features that characterize a benchmarking tool (workload representation, load generation, data collection, output analysis and report) and their applicability to the analysis of distributed Web-server systems.

[1]  Srinivasan Seshan,et al.  The effects of wide-area conditions on WWW server performance , 2001, SIGMETRICS '01.

[2]  Zhen Liu,et al.  Traffic model and performance evaluation of Web servers , 2001, Perform. Evaluation.

[3]  Daniel A. Menascé,et al.  Scaling for E-Business: Technologies, Models, Performance, and Capacity Planning , 2000 .

[4]  Balachander Krishnamurthy,et al.  Analyzing factors that influence end-to-end Web performance , 2000, Comput. Networks.

[5]  Roy T. Fielding,et al.  Hypertext Transfer Protocol - HTTP/1.1 , 1997, RFC.

[6]  Srinivasan Seshan,et al.  TCP behavior of a busy Internet server: analysis and improvements , 1997, Proceedings. IEEE INFOCOM '98, the Conference on Computer Communications. Seventeenth Annual Joint Conference of the IEEE Computer and Communications Societies. Gateway to the 21st Century (Cat. No.98.

[7]  Imrich Chlamtac,et al.  The P2 algorithm for dynamic calculation of quantiles and histograms without storing observations , 1985, CACM.

[8]  Mon-Yen Luo,et al.  A content placement and management system for distributed Web-server systems , 2000, Proceedings 20th IEEE International Conference on Distributed Computing Systems.

[9]  Carey L. Williamson,et al.  A case study of Web server benchmarking using parallel WAN emulation , 2002, Perform. Evaluation.

[10]  Daniel A. Menascé,et al.  TPC-W: A Benchmark for E-Commerce , 2002, IEEE Internet Comput..

[11]  D.A. Menasce,et al.  Scaling for e-business , 2000, Proceedings 8th International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (Cat. No.PR00728).

[12]  Philip S. Yu,et al.  Geographic load balancing for scalable distributed Web systems , 2000, Proceedings 8th International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (Cat. No.PR00728).

[13]  Walter Willinger,et al.  Self-similarity through high-variability: statistical analysis of Ethernet LAN traffic at the source level , 1997, TNET.

[14]  Peter Druschel,et al.  Measuring the capacity of a Web server under realistic loads , 1999, World Wide Web.

[15]  Margo I. Seltzer,et al.  A self-scaling and self-configuring benchmark for Web servers (extended abstract) , 1998, SIGMETRICS '98/PERFORMANCE '98.

[16]  Paul Barford,et al.  A performance evaluation of hyper text transfer protocols , 1999, SIGMETRICS '99.

[17]  Paul Barford,et al.  Generating representative Web workloads for network and server performance evaluation , 1998, SIGMETRICS '98/PERFORMANCE '98.

[18]  Balachander Krishnamurthy,et al.  Key Differences Between HTTP/1.0 and HTTP/1.1 , 1999, Comput. Networks.

[19]  Martin Arlitt,et al.  A workload characterization study of the 1998 World Cup Web site , 2000, IEEE Netw..

[20]  Jerome A. Rolia,et al.  Characterizing the scalability of a large web-based shopping system , 2001, ACM Trans. Internet Techn..

[21]  Jerome A. Rolia,et al.  Predicting the QoS of an electronic commerce server: those mean percentiles , 1998, PERV.

[22]  Carey L. Williamson,et al.  Internet Web servers: workload characterization and performance implications , 1997, TNET.

[23]  Philip S. Yu,et al.  The state of the art in locally distributed Web-server systems , 2002, CSUR.

[24]  D. B. Davis,et al.  Intel Corp. , 1993 .

[25]  Balachander Krishnamurthy,et al.  Web Protocols and Practice - HTTP/1.1, Networking Protocols, Caching, and Traffic Measurement , 2001 .

[26]  David Mosberger,et al.  httperf—a tool for measuring web server performance , 1998, PERV.

[27]  Carey L. Williamson,et al.  Web server benchmarking using parallel WAN emulation , 2002, SIGMETRICS '02.

[28]  Azer Bestavros,et al.  Self-similarity in World Wide Web traffic: evidence and possible causes , 1997, TNET.

[29]  Martin F. Arlitt,et al.  Characterizing Web user sessions , 2000, PERV.

[30]  Virgílio A. F. Almeida,et al.  Capacity Planning for Web Services: Metrics, Models, and Methods , 2001 .

[31]  Raj Jain,et al.  The art of computer systems performance analysis - techniques for experimental design, measurement, simulation, and modeling , 1991, Wiley professional computing.

[32]  Virgílio A. F. Almeida.  Capacity Planning for Web Services , 2002, Performance.

[33]  Mark S. Squillante,et al.  Analysis and characterization of large‐scale Web server access patterns and performance , 1999, World Wide Web.

[34]  David Finkel,et al.  Book review: The Art of Computer Systems Performance Analysis by R. Jain (Wiley-Interscience, 1991) , 1990, PERV.

[35]  Luigi Rizzo,et al.  Dummynet: a simple approach to the evaluation of network protocols , 1997, CCRV.

[36]  Paul Barford,et al.  Critical path analysis of TCP transactions , 2001, TNET.