A whitepaper from iAnywhere Solutions, Inc., a subsidiary of Sybase Inc. Capacity planning with SQL Anywhere

We present an overview of issues to consider when designing a performance evaluation of sql Anywhere. In particular, we outline potential pitfalls when designing a benchmark based on a specific database application. We address issues related both to the construction of a model database instance, and a model application workload, and outline an approach to construct a model of performance to determine the relative importance of various performance factors, such as database cache size and the server multiprogramming level. The paper contains several examples illustrating the potential of negative scalability that can occur when executing concurrent requests in an smp environment, and how these scenarios can be generated easily by artificial contention caused by poorly-defined workloads.

[1]  Meikel Pöss,et al.  TPC-DS, taking decision support benchmarking to the next level , 2002, SIGMOD '02.

[2]  Ramesh Bhashyam.  TPC-D—the challenges, issues and results , 1996, SGMD.

[3]  Patrick Martin,et al.  Sizing DB2 UDB® servers for business intelligence workloads , 2004, CASCON.

[4]  Luiz André Barroso,et al.  Memory system characterization of commercial workloads , 1998, ISCA.

[5]  Giuseppe Serazzi,et al.  Measurement and Tuning of Computer Systems , 1984, Int. CMG Conference.

[6]  Virgílio A. F. Almeida,et al.  Capacity Planning for Web Services: Metrics, Models, and Methods , 2001 .

[7]  Roderic G. G. Cattell.  The benchmark handbook for database and transaction processing systems , 1991 .

[8]  John Paul Shen,et al.  Scaling and characterizing database workloads: bridging the gap between research and practice , 2003, Proceedings. 36th Annual IEEE/ACM International Symposium on Microarchitecture, 2003. MICRO-36..

[9]  S. Parekh,et al.  An analysis of database workload performance on simultaneous multithreaded processors , 1998, Proceedings. 25th Annual International Symposium on Computer Architecture (Cat. No.98CB36235).

[10]  Sarita V. Adve,et al.  Performance of database workloads on shared-memory systems with out-of-order processors , 1998, ASPLOS VIII.

[11]  Stavros Christodoulakis,et al.  Implications of certain assumptions in database performance evauation , 1984, TODS.

[12]  H. J. Arnold.  Introduction to the Practice of Statistics , 1990 .

[13]  Winfried Materna.  An Approach to the Construction of Workload Models , 1979, Performance.

[14]  Paul J. Fortier,et al.  Computer Systems Performance Evaluation and Prediction , 2003 .

[15]  Dror G. Feitelson,et al.  Workload Modeling for Performance Evaluation , 2002, Performance.

[16]  Domenico Ferrari,et al.  Computer Systems Performance Evaluation , 1978 .

[17]  David K. Hsiao,et al.  Design Analysis and Performance Evaluation Methodologies for Database Computers , 1987, Adv. Comput..

[18]  Ann Marie Grizzaffi Maynard,et al.  Contrasting characteristics and cache performance of technical and multi-user commercial workloads , 1994, ASPLOS VI.

[19]  Ludmila Svobodová.  Computer Performance Measurement and Evaluation Methods: Analysis and Applications. , 1974 .

[20]  David J. DeWitt,et al.  A performance analysis of the gamma database machine , 1988, SIGMOD '88.

[21]  David J. DeWitt,et al.  Benchmarking Database Systems A Systematic Approach , 1983, VLDB.

[22]  David J. DeWitt,et al.  A methodology for database system performance evaluation , 1984, SIGMOD '84.

[23]  David A. Patterson,et al.  A new approach to I/O performance evaluation: self-scaling I/O benchmarks, predicted I/O performance , 1994, TOCS.

[24]  David B. Skillicorn,et al.  Developing a characterization of business intelligence workloads for sizing new database systems , 2004, DOLAP '04.

[25]  G. C. Steindel,et al.  A Benchmark Comparison of DB2 and the DBC/1012 , 1989, Computer Measurement Group Transactions.

[26]  Patrick Martin,et al.  A Study of Capacity Planning for Database Management Systems with OLAP Workloads , 2003, Int. CMG Conference.

[27]  Ray Jain,et al.  The art of computer systems performance analysis - techniques for experimental design, measurement, simulation, and modeling , 1991, Wiley professional computing.

[28]  David K. Hsiao,et al.  Performance measurement methodologies for database systems , 1985, ACM '85.

[29]  Michael Stonebraker,et al.  Performance analysis of a relational data base management system , 1979, SIGMOD '79.

[30]  Mandyam M. Srinivasan,et al.  Introduction To Computer System Performance Evaluation , 1992 .