An infrastructure for generating and sharing experimental workloads for persistent object systems

Performance evaluation of persistent object system implementations requires the use and evaluation of experimental workloads. Such workloads include a schema describing how the data are related, and application behaviors that capture how the data are manipulated over time. In this paper, we describe an infrastructure for generating and sharing experimental workloads to be used in evaluating the performance of persistent object system implementations. The infrastructure consists of a toolkit that aids the analyst in modeling and instrumenting experimental workloads, and a trace format that allows the analyst to easily reuse and share the workloads. Our infrastructure provides the following benefits: the process of building new experiments for analysis is made easier; experiments to evaluate the performance of implementations can be conducted and reproduced with less effort; and pertinent information can be gathered in a cost‐effective manner. We describe the two major components of this infrastructure, the trace format and the toolkit. We also describe our experiences using these components to model, instrument, and experiment with the OO7 benchmark. Copyright © 2000 John Wiley & Sons, Ltd.

[1]  A. Dain Samples,et al.  Mache: no-loss trace compaction , 1989, SIGMETRICS '89.

[2]  Frank Yellin,et al.  The Java Virtual Machine Specification , 1996 .

[3]  David Jordan,et al.  The Object Database Standard: ODMG 2.0 , 1997 .

[4]  Jeffrey F. Naughton,et al.  Storage reclamation and reorganization in client-server persistent object stores , 1994, Proceedings of 1994 IEEE 10th International Conference on Data Engineering.

[5]  Benjamin G. Zorn.  The Effect of Garbage Collection on Cache Performance , 1991 .

[6]  David J. DeWitt,et al.  The oo7 Benchmark , 1993, SIGMOD Conference.

[7]  Henry M. Levy,et al.  Evaluation of OO7 as a system and an application benchmark , 1995 .

[8]  Malcolm P. Atkinson,et al.  PS-algol: an algol with a persistent heap , 1982, SIGP.

[9]  Roger King,et al.  Self-adaptive, on-line reclustering of complex object data , 1994, SIGMOD '94.

[10]  Jim Gray,et al.  Benchmark Handbook: For Database and Transaction Processing Systems , 1992 .

[11]  Trevor N. Mudge,et al.  Trace-driven memory simulation: a survey , 1997, CSUR.

[12]  David S. Munro,et al.  The MaStA I/O Cost Model and its Validation Strategy , 1995, ADBIS.

[13]  Laurent Amsaleg,et al.  Evaluating Garbage Collectors for Large Persistence Stores , 1995, OOPSLA 1995.

[14]  Alexander L. Wolf,et al.  Semi-automatic, self-adaptive control of garbage collection rates in object databases , 1996, SIGMOD '96.

[15]  Barbara Liskov,et al.  Partitioned garbage collection of a large object store , 1997, SIGMOD '97.

[16]  Michele Missikoff,et al.  MOSAICO—a system for conceptual modeling and rapid prototyping of object-oriented database application , 1994, SIGMOD '94.

[17]  T. Lougenia Anderson,et al.  The HyperModel Benchmark , 1990, EDBT.

[18]  Alexander L. Wolf,et al.  A Highly Effective Partition Selection Policy for Object Database Garbage Collection , 1998, IEEE Trans. Knowl. Data Eng..

[19]  Mark A. Holliday.  Techniques for Cache and Memory Simulation Using Address Reference Traces , 1991, Int. J. Comput. Simul..

[20]  Alexander L. Wolf,et al.  Partition selection policies in object database garbage collection , 1994, SIGMOD '94.

[21]  Laurent Amsaleg,et al.  Efficient incremental garbage collection for workstation/server database systems , 1994 .