Automatic Generation of Precise and Useful Commutativity Conditions (Extended Version)

Reasoning about commutativity between data-structure operations is an important problem with applications including parallelizing compilers, optimistic parallelization and, more recently, Ethereum smart contracts. There have been research results on automatic generation of commutativity conditions, yet we are unaware of any fully automated technique to generate conditions that are both sound and effective. We have designed such a technique, driven by an algorithm that iteratively refines a conservative approximation of the commutativity (and non-commutativity) condition for a pair of methods into an increasingly precise version. The algorithm terminates if/when the entire state space has been considered, and can be aborted at any time to obtain a partial yet sound commutativity condition. We have generalized our work to left-/right-movers and proved relative completeness. We describe aspects of our technique that lead to useful commutativity conditions, including how predicates are selected during refinement and heuristics that impact the output shape of the condition. We have implemented our technique in a prototype open-source tool Servois. Our algorithm produces quantifier-free queries that are dispatched to a back-end SMT solver. We evaluate Servois through two case studies: (i) We synthesize commutativity conditions for a range of data structures including Set, HashTable, Accumulator, Counter, and Stack. (ii) We consider an Ethereum smart contract called BlockKing, and show that Servois can detect serious concurrency-related vulnerabilities and guide developers to construct robust and efficient implementations.

[1]  Martín Abadi,et al.  The existence of refinement mappings , 1988, [1988] Proceedings. Third Annual Information Symposium on Logic in Computer Science.

[2]  Bertrand Meyer,et al.  Applying 'design by contract' , 1992, Computer.

[3]  L. Flon.  A Unified Approach to the Specification and Verification of Abstract DataTypes , 1978 .

[4]  C. A. R. Hoare,et al.  Proof of correctness of data representations , 1972, Acta Informatica.

[5]  Benjamin Goldberg,et al.  Theory and algorithms for the generation and validation of speculative loop optimizations , 2004, Proceedings of the Second International Conference on Software Engineering and Formal Methods, 2004. SEFM 2004..

[6]  William F. Ogden,et al.  Specification of Abstract Data Types in Modula , 1980, TOPL.

[7]  K. Rustan M. Leino.  Specifying and Verifying Programs in Spec# , 2006, Ershov Memorial Conference.

[8]  Nathan Clark,et al.  Commutativity analysis for software parallelization: letting program transformations see the big picture , 2009, ASPLOS.

[9]  Maurice Herlihy,et al.  Transactional boosting: a methodology for highly-concurrent transactional objects , 2008, PPoPP.

[10]  Maurice Herlihy,et al.  Coarse-grained transactions , 2010, POPL '10.

[11]  Edmund M. Clarke,et al.  Counterexample-guided abstraction refinement , 2003, 10th International Symposium on Temporal Representation and Reasoning, 2003 and Fourth International Conference on Temporal Logic. Proceedings..

[12]  Dimitar Dimitrov,et al.  Commutativity race detection , 2014, PLDI.

[13]  Chao Wang,et al.  Peephole Partial Order Reduction , 2008, TACAS.

[14]  Dimitar Dimitrov,et al.  Learning Commutativity Specifications , 2015, CAV.

[15]  Cesare Tinelli,et al.  A New Decision Procedure for Finite Sets and Cardinality Constraints in SMT , 2016, IJCAR.

[16]  Software Pioneers , 2002, Springer Berlin Heidelberg.

[17]  Roman Manevich,et al.  JANUS: exploiting parallelism via hindsight , 2012, PLDI '12.

[18]  Eran Yahav,et al.  Abstraction-guided synthesis of synchronization , 2010, POPL '10.

[19]  Maurice Herlihy,et al.  Adding Concurrency to Smart Contracts , 2017, PODC.

[20]  Shmuel Sagiv,et al.  HAWKEYE: effective discovery of dataflow impediments to parallelization , 2011, OOPSLA '11.

[21]  Austin T. Clements,et al.  The scalable commutativity rule: designing scalable software for multicore processors , 2013, SOSP.

[22]  Rajeev Alur,et al.  A Temporal Logic of Nested Calls and Returns , 2004, TACAS.

[23]  Eran Yahav,et al.  Deriving linearizable fine-grained concurrent objects , 2008, PLDI '08.

[24]  K. Rustan M. Leino,et al.  The Spec# Programming System: An Overview , 2004, CASSIS.

[25]  Eric Koskinen,et al.  Making prophecies with decision predicates , 2011, POPL '11.

[26]  Martin C. Rinard,et al.  Verification of semantic commutativity conditions and inverse operations on linked data structures , 2011, PLDI '11.

[27]  Ilya Sergey,et al.  A Concurrent Perspective on Smart Contracts , 2017, Financial Cryptography Workshops.

[28]  Martin C. Rinard,et al.  Commutativity analysis: a new analysis technique for parallelizing compilers , 1997, TOPL.

[29]  Richard J. Lipton,et al.  Reduction: a method of proving properties of parallel programs , 1975, CACM.

[30]  Armando Solar-Lezama,et al.  Sketching concurrent data structures , 2008, PLDI '08.

[31]  Kshitij Bansal,et al.  Decision Procedures for Finite Sets with Cardinality, and Local Theories Extensions , 2016 .

[32]  Bratin Saha,et al.  Open nesting in software transactional memory , 2007, PPOPP.

[33]  Matthew J. Parkinson,et al.  The Push/Pull model of transactions , 2015, PLDI.

[34]  Keshav Pingali,et al.  Exploiting the commutativity lattice , 2011, PLDI '11.