Requirements-driven database systems benchmark method

Benchmarks are the vital tools in the performance measurement, evaluation, and comparison of relational database management systems (RDBMS). Standard benchmarks such as the TP1, TPC-A, TPC-B, TPC-C, TPC-D, TPC-H, TPC-R, TPC-W, Wisconsin, and AS3 Ap benchmarks have been used to assess the performance of relational database management systems. These benchmarks are synthetic and domain-specific. Test results from these benchmarks are estimates of possible system performance for certain pre-determined application types. Database system performance on actual database domain may vary significantly from those in the standard benchmarks. In this paper, we describe a new benchmark method that is computer-assisted and developed from the perspective of the user's requirements.

[1]  Carolyn Turbyfill,et al.  AS3AP - A Comparative Relational Database Benchmark , 1989 .

[2]  Domenico Ferrari,et al.  Performance analysis of several back-end database architectures , 1986, TODS.

[3]  Domenico Ferrari,et al.  Computer Systems Performance Evaluation , 1978 .

[4]  Philip J. Fleming,et al.  How not to lie with statistics: the correct way to summarize benchmark results , 1986, CACM.

[5]  Wilbur H. Highleyman.  Performance Analysis of Transaction Processing Systems , 1989, SIGMETRICS Perform. Evaluation Rev..

[6]  Alfonso F. Cardenas,et al.  Evaluation and selection of file organization—a model and system , 1973, Commun. ACM.

[7]  Ashvini Nangia.  Book Review: Performance Analysis of Transaction Processing Systems by Wilbur H. Highleyman (Prentice Hall, 1989) , 1992, PERV.

[8]  C. Turbyfill.  Comparative Benchmarking of Relational Database Systems , 1988 .

[9]  David J. DeWitt,et al.  Benchmarking Database Systems A Systematic Approach , 1983, VLDB.

[10]  Per-Ake Larson,et al.  Proceedings of the 1988 ACM SIGMOD International Conference on Management of Data, Chicago, Illinois, USA, June 1-3, 1988 , 1988, SIGMOD 1988.

[11]  Jim Gray,et al.  Benchmark Handbook: For Database and Transaction Processing Systems , 1992 .

[12]  Alan R. Hevner,et al.  Analysis of Database System Architectures Using Benchmarks , 1987, IEEE Transactions on Software Engineering.

[13]  Beatrice Yormark,et al.  Proceedings of the 1984 ACM SIGMOD international conference on Management of data , 1977 .

[14]  Philip S. Yu,et al.  On Workload Characterization of Relational Database Environments , 1992, IEEE Trans. Software Eng..

[15]  Sushil Jajodia,et al.  Proceedings of the 1993 ACM SIGMOD international conference on Management of data , 1993, SIGMOD 1993.

[16]  David J. DeWitt,et al.  A performance analysis of the gamma database machine , 1988, SIGMOD '88.

[17]  David J. DeWitt,et al.  A methodology for database system performance evaluation , 1984, SIGMOD '84.

[18]  Philip Heidelberger,et al.  Computer Performance Evaluation Methodology , 1984, IEEE Transactions on Computers.

[19]  Michael Stonebraker,et al.  A measure of transaction processing power , 1985 .

[20]  Daniel M. Dias,et al.  A modeling study of the TPC-C benchmark , 1993, SIGMOD '93.

[21]  Carolyn Turbyfill,et al.  AS 3 AP: An ANSI SQL Standard Scaleable and Portable Benchmark for Relational Database Systems. , 1991 .

[22]  Connolly,et al.  Database Systems , 2004 .