Performance study on logging to another main memory database

In update intensive main memory database applications, huge volume of log records is generated, to maintain the ACID properties of the database system, the log records should be persistent efficiently. Delegating logging of one main memory database to another main memory database is proposed. The scheme is elaborated in detail in terms of architecture, logging & safeness levels, checkpointing, and recovery. Strict durability and relax durability are provided. When some form of non-volatile memory is used to temporarily holding log records, not only logging efficiency is improved, but also the scheme could guarantee full ACID of the system. We also propose using parallel logging to speedup log persistence by writing logs to multiple disks in parallel. Since interconnection network techniques progress by leaps and bounds, the scheme eliminates the concern about whether the system's overall performance may be slowed down by bandwidth and latency limitations. Experiment results demonstrate the feasibility of the proposal.

[1]  Henry F. Korth,et al.  An Almost-Serial Protocol for Transaction Execution in Main-Memory Database Systems , 2002, VLDB.

[2]  Wei Hu,et al.  Architecture of Highly Available Databases , 2004, ISAS.

[3]  Hideyuki Kawashima,et al.  Providing Persistence for Sensor Data Streams by Remote WAL , 2006, DaWaK.

[4]  Kihong Kim,et al.  Differential logging: a commutative and associative logging scheme for highly parallel main memory database , 2001, Proceedings 17th International Conference on Data Engineering.

[5]  K. M. Chandy,et al.  Incremental Recovery In Main Memory Database Systems , 1992 .

[6]  Hideyuki Kawashima,et al.  Accelerating Remote Logging by Two Level Asynchronous Checkpointing , 2006, 7th International Conference on Mobile Data Management (MDM'06).

[7]  Haengrae Cho,et al.  Checkpointing schemes for fast restart in main memory database systems , 1997, 1997 IEEE Pacific Rim Conference on Communications, Computers and Signal Processing, PACRIM. 10 Years Networking the Pacific Rim, 1987-1997.

[8]  Michael Wu,et al.  eNVy: a non-volatile, main memory storage system , 1994, ASPLOS VI.

[9]  S. Sudarshan,et al.  Recovering from Main-Memory Lapses , 1993, VLDB.

[10]  Wang Shan,et al.  A Parallel Recovery Scheme for Update Intensive Main Memory Database Systems , 2008, 2008 Ninth International Conference on Parallel and Distributed Computing, Applications and Technologies.

[11]  Antoni Wolski,et al.  Performance Measurement and Tuning of Hot-Standby Databases , 2006, ISAS.

[12]  Hideyuki Kawashima,et al.  Providing persistence for sensor streams with light neighbor WAL , 2002, 2002 Pacific Rim International Symposium on Dependable Computing, 2002. Proceedings..

[13]  Sang Kyun Cha,et al.  P*TIME: Highly Scalable OLTP DBMS for Managing Update-Intensive Stream Workload , 2004, VLDB.

[14]  Svein-Olaf Hvasshovd,et al.  The ClustRa Telecom Database: High Availability, High Throughput, and Real-Time Response , 1995, VLDB.