DERBY: a memory management system for distributed main memory databases

This paper describes a main memory data storage system for a distributed system of heterogeneous general purpose workstations. We show that distributed main memory storage managers are qualitatively different from distributed disk-based storage managers. Specifically, we show that load balancing, which is crucial to disk-based systems, has little effect on the performance of a memory-based system. On the other hand, we show that saturation prevention in cases where the server exceeds its memory capacity or becomes overloaded is crucial to smooth performance. Finally, we show that distributed memory-based storage results in performance more than one order of magnitude better than their disk-based counterparts.

[1]  Michael J. Carey,et al.  A recovery algorithm for a high-performance memory-resident database system , 1987, SIGMOD '87.

[2]  M. Franklin,et al.  Global Memory Management in Client-Server DBMS Architectures , 1992 .

[3]  Willy Zwaenepoel,et al.  Implementation and performance of Munin , 1991, SOSP '91.

[4]  Andrew R. Cherenson,et al.  The Sprite network operating system , 1988, Computer.

[5]  Michael J. Carey,et al.  A Study of Index Structures for a Main Memory Database Management System , 1986, HPTS.

[6]  Hector Garcia-Molina,et al.  System M: A Transaction Processing Testbed for Memory Resident Data , 1990, IEEE Trans. Knowl. Data Eng..

[7]  S. Sudarshan,et al.  DataBlitz: A High Performance Main-Memory Storage Manager , 1994, VLDB.

[8]  Gerhard Weikum,et al.  Distributed file organization with scalable cost/performance , 1994, SIGMOD '94.

[9]  Paul Hudak,et al.  Memory coherence in shared virtual memory systems , 1989, TOCS.

[10]  Gerhard Weikum,et al.  The LRU-K page replacement algorithm for database disk buffering , 1993, SIGMOD Conference.

[11]  S. Sudarshan,et al.  Dalí: A High Performance Main Memory Storage Manager , 1994, VLDB.

[12]  Brian N. Bershad,et al.  The Midway distributed shared memory system , 1993, Digest of Papers. Compcon Spring.

[13]  Fred Douglis,et al.  Process Migration in the Sprite Operating System , 1987, ICDCS.

[14]  Liuba Shrira,et al.  Escaping the disk bottleneck in fast transaction processing , 1992, [1992] Proceedings Third Workshop on Workstation Operating Systems.

[15]  Kemal Efe,et al.  Optimal Scheduling of Compute-Intensive Tasks on a Network of Workstations , 1995, IEEE Trans. Parallel Distributed Syst..

[16]  Michael J. Carey,et al.  Client-Server Caching Revisited , 1998, IWDOM.

[17]  Garcia-MolinaH.,et al.  Main Memory Database Systems , 1992 .

[18]  Michael Stonebraker,et al.  Implementation techniques for main memory database systems , 1984, SIGMOD '84.

[19]  L EagerDerek,et al.  Adaptive load sharing in homogeneous distributed systems , 1986 .

[20]  Gary Scott Delp.  The architecture and implementation of MEMNET: a high--speed shared-memory computer communication network , 1988 .

[21]  Hector Garcia-Molina,et al.  Main Memory Database Systems: An Overview , 1992, IEEE Trans. Knowl. Data Eng..

[22]  Kenneth Salem,et al.  Management of Partially Safe Buffers , 1995, IEEE Trans. Computers.

[23]  Edward D. Lazowska,et al.  Adaptive load sharing in homogeneous distributed systems , 1986, IEEE Transactions on Software Engineering.

[24]  Miron Livny,et al.  Condor-a hunter of idle workstations , 1988, [1988] Proceedings. The 8th International Conference on Distributed.

[25]  Miron Livny,et al.  Global Memory Management in Client-Server Database Architectures , 1992, VLDB.

[26]  Umakishore Ramachandran,et al.  An implementation of distributed shared memory , 1991, Softw. Pract. Exp..