Research and implementation of a distributed transaction processing middleware

Abstract Currently, increasingly transactional requests require high-performance transaction processing systems as support. The performance of a distributed transaction processing system is superior to that of traditional single-node transaction processing system, and the characteristic of multi-node determines that distributed transaction processing systems should pay more attention to availability. For example, in traditional single-node systems, the performance of Berkeley DB is high, but its shortcoming of not supporting parallel writing across multiple nodes is weakening its availability and scalability in the distributed environment. This paper has designed and implemented a middleware-level distributed transaction processing system called POST, including a distributed database system called POSTBOX which is based on Berkeley DB and data partition, and a distributed transaction processing middleware called POSTMAN. POSTBOX inherits the availability of highly available Berkeley DB, and expands it with data partition. By Partition Replication Body (PRB), POSTBOX overcomes the native weakness of highly available Berkeley DB, which indicates that highly available Berkeley DB does not support parallel writing across multiple nodes; POSTMAN is a middleware adapting PRB. POSTMAN monitors POSTBOX in real-time via Partition Replication Body State Array (PRBSA), and ensures the correctness of transaction processing and transactions migration in the case of node failure. The actual test results show that POST possesses high availability, and has an obvious improvement of write performance compared with highly available Berkeley DB.

[1]  Albert Y. Zomaya,et al.  Task-Tree Based Large-Scale Mosaicking for Massive Remote Sensed Imageries with Dynamic DAG Scheduling , 2014, IEEE Transactions on Parallel and Distributed Systems.

[2]  Jiwu Shu,et al.  Keyword search with access control over encrypted data in cloud computing , 2014, 2014 IEEE 22nd International Symposium of Quality of Service (IWQoS).

[3]  Wolfgang Emmerich,et al.  The Impact of Research on Middleware Technology , 2007, 29th International Conference on Software Engineering (ICSE'07 Companion).

[4]  Lizhe Wang,et al.  DDDAS-Based Parallel Simulation of Threat Management for Urban Water Distribution Systems , 2014, Computing in Science & Engineering.

[5]  E. Brewer,et al.  CAP twelve years later: How the "rules" have changed , 2012, Computer.

[6]  Giuseppe Pelagatti,et al.  Formal semantics of SQL queries , 1991, TODS.

[7]  Keqiang He,et al.  Next stop, the cloud: understanding modern web service deployment in EC2 and azure , 2013, Internet Measurement Conference.

[8]  Nancy A. Lynch,et al.  Brewer's conjecture and the feasibility of consistent, available, partition-tolerant web services , 2002, SIGA.

[9]  Jiwu Shu,et al.  Secure storage system and key technologies , 2013, 2013 18th Asia and South Pacific Design Automation Conference (ASP-DAC).

[10]  Wei Xue,et al.  A case study of large-scale parallel I/O analysis and optimization for numerical weather prediction system , 2014, Future Gener. Comput. Syst..

[11]  Ricardo Jiménez-Peris,et al.  Middleware based data replication providing snapshot isolation , 2005, SIGMOD '05.

[12]  Fernando Pedone,et al.  Sprint: a middleware for high-performance transaction processing , 2007, EuroSys '07.

[13]  Randi Karlsen,et al.  Flexible transaction processing in the Argos middleware , 2008, SETMDM '08.

[14]  Rajiv Ranjan,et al.  Towards building a data-intensive index for big data computing - A case study of Remote Sensing data processing , 2015, Inf. Sci..

[15]  Bhushan Nemade,et al.  Cloud computing: Windows Azure platform , 2011, ICWET.

[16]  Michael Burrows,et al.  The Chubby Lock Service for Loosely-Coupled Distributed Systems , 2006, OSDI.

[17]  M. Tamer Özsu,et al.  Using semantic knowledge of transactions to increase concurrency , 1989, TODS.

[18]  Stanley B. Zdonik,et al.  On Predictive Modeling for Optimizing Transaction Execution in Parallel OLTP Systems , 2011, Proc. VLDB Endow..

[19]  Michael Stonebraker,et al.  New opportunities for New SQL , 2012, CACM.

[20]  Michael Stonebraker,et al.  H-store: a high-performance, distributed main memory transaction processing system , 2008, Proc. VLDB Endow..

[21]  Albert Y. Zomaya,et al.  Particle Swarm Optimization based dictionary learning for remote sensing big data , 2015, Knowl. Based Syst..

[22]  Ehud Gudes,et al.  TOPS: a new design for transactions in publish/subscribe middleware , 2008, DEBS.

[23]  Pradeep Dubey,et al.  Fast Updates on Read-Optimized Databases Using Multi-Core CPUs , 2011, Proc. VLDB Endow..

[24]  Luciana Arantes,et al.  Facing peak loads in a P2P transaction system , 2012, P2P-Dep '12.

[25]  Michael Stonebraker,et al.  The VoltDB Main Memory DBMS , 2013, IEEE Data Eng. Bull..

[26]  Albert Y. Zomaya,et al.  Parallel Processing of Dynamic Continuous Queries over Streaming Data Flows , 2015, IEEE Transactions on Parallel and Distributed Systems.

[27]  Jiwu Shu,et al.  Shield: A stackable secure storage system for file sharing in public storage , 2014, J. Parallel Distributed Comput..

[28]  Wilson C. Hsieh,et al.  Bigtable: A Distributed Storage System for Structured Data , 2006, TOCS.

[29]  Michael Stonebraker,et al.  SciDB DBMS Research at M.I.T , 2013, IEEE Data Eng. Bull..

[30]  Hubert Naacke,et al.  TransPeer: adaptive distributed transaction monitoring for Web2.0 applications , 2010, SAC '10.

[31]  Jiwu Shu,et al.  Preferred keyword search over encrypted data in cloud computing , 2013, 2013 IEEE/ACM 21st International Symposium on Quality of Service (IWQoS).

[32]  Albert Y. Zomaya,et al.  A Parallel File System with Application-Aware Data Layout Policies for Massive Remote Sensing Image Processing in Digital Earth , 2015, IEEE Transactions on Parallel and Distributed Systems.

[33]  Yang Liu,et al.  Corslet: A shared storage system keeping your data private , 2011, Science China Information Sciences.