Reordering Transaction Execution to Boost High-Frequency Trading Applications

High frequency trading (HFT) has always been welcomed because it benefits not only personal interests but also the whole social welfare. While the recent advance of portfolio selection in HFT market generates more profit, it yields much contended OLTP workloads. Featuring in exploiting the abundant parallelism, transaction pipeline, the state-of-the-art concurrency control (CC) mechanism, however suffers from limited concurrency confronted with HFT workloads. Its variants that enable more parallel execution by leveraging find-grained contention information also take little effect. To solve this problem, we for the first time observe and formulate the source of restricted concurrency as harmful ordering of transaction statements. To resolve harmful ordering, we propose PARE, a pipeline-aware reordered execution, to improve application performance by rearranging statements in order of their degrees of contention. In concrete, two mechanisms are devised to ensure the correctness of statement rearrangement and identify the degrees of contention of statements respectively. Experiment results show that PARE can improve transaction throughput and reduce transaction latency on HFT applications by up to an order of magnitude than the state-of-the-art CC mechanism.

[1]  Ippokratis Pandis,et al.  Data-oriented transaction execution , 2010, Proc. VLDB Endow..

[2]  Ji-Rong Wen,et al.  Discovering Neighborhood Pattern Queries by sample answers in knowledge base , 2016, 2016 IEEE 32nd International Conference on Data Engineering (ICDE).

[3]  Michael Wolfe,et al.  High performance compilers for parallel computing , 1995 .

[4]  Yoav Freund,et al.  Automated trading with boosting and expert weighting , 2010 .

[5]  Ali Ghodsi,et al.  Coordination Avoidance in Database Systems , 2014, Proc. VLDB Endow..

[6]  Ji-Rong Wen,et al.  Mining frequent neighborhood patterns in a large labeled graph , 2013, CIKM.

[7]  Barbara G. Ryder,et al.  Properties of data flow frameworks , 1990, Acta Informatica.

[8]  Gottfried Vossen,et al.  Transactional Information Systems: Theory, Algorithms, and the Practice of Concurrency Control and Recovery , 2002 .

[9]  Daniel J. Abadi,et al.  High Performance Transactions via Early Write Visibility , 2017, Proc. VLDB Endow..

[10]  Kian-Lee Tan,et al.  Transaction Healing: Scaling Optimistic Concurrency Control on Multicores , 2016, SIGMOD Conference.

[11]  Amir Shaikhha,et al.  Repairing Conflicts among MVCC Transactions , 2016, ArXiv.

[12]  Alfons Kemper,et al.  Fast Serializable Multi-Version Concurrency Control for Main-Memory Database Systems , 2015, SIGMOD Conference.

[13]  Alvin Cheung,et al.  Automatic Partitioning of Database Applications , 2012, Proc. VLDB Endow..

[14]  Jun Wang,et al.  Portfolio Choices with Orthogonal Bandit Learning , 2015, IJCAI.

[15]  Haibo Chen,et al.  Scaling Multicore Databases via Constrained Parallel Execution , 2016, SIGMOD Conference.

[16]  J. T. Robinson,et al.  On optimistic methods for concurrency control , 1979, TODS.

[17]  Jian Pei,et al.  Within-Network Classification Using Radius-Constrained Neighborhood Patterns , 2014, CIKM.

[18]  Alvin Cheung,et al.  Leveraging Lock Contention to Improve OLTP Application Performance , 2016, Proc. VLDB Endow..

[19]  Jignesh M. Patel,et al.  High-Performance Concurrency Control Mechanisms for Main-Memory Databases , 2011, Proc. VLDB Endow..

[20]  Irving L. Traiger,et al.  The notions of consistency and predicate locks in a database system , 1976, CACM.

[21]  Gerhard Weikum,et al.  CHAPTER FOUR – Concurrency Control Algorithms , 2002 .

[22]  Marc H. Scholl,et al.  Transactional information systems: theory, algorithms, and the practice of concurrency control and recovery , 2001, SGMD.

[23]  Alvin Cheung,et al.  Sloth: being lazy is a virtue (when issuing database queries) , 2014, SIGMOD Conference.