Cobra: Making Transactional Key-Value Stores Verifiably Serializable

Today’s cloud databases offer strong properties, including serializability, sometimes called the gold standard database correctness property. But cloud databases are complicated black boxes, running in a different administrative domain from their clients. Thus, clients might like to know whether the databases are meeting their contract. To that end, we introduce cobra; cobra applies to transactional key-value stores. It is the first system that combines (a) black-box checking, of (b) serializability, while (c) scaling to real-world online transactional processing workloads. The core technical challenge is that the underlying search problem is computationally expensive. Cobra tames that problem by starting with a suitable SMT solver. Cobra then introduces several new techniques, including a new encoding of the validity condition; hardware acceleration to prune inputs to the solver; and a transaction segmentation mechanism that enables scaling and garbage collection. Cobra imposes modest overhead on clients, improves over baselines by 10× in verification cost, and (unlike the baselines) supports continuous verification. Our artifact can handle 2000 transactions/sec, equivalent to 170M/day.

[1]  Xin-She Yang,et al.  Introduction to Algorithms , 2021, Nature-Inspired Optimization Algorithms.

[2]  Peter Alvaro,et al.  Elle: Inferring Isolation Anomalies from Experimental Observations , 2020, Proc. VLDB Endow..

[3]  Ahmed Awad,et al.  Execution integrity without implicit trust of system software , 2019, SysTEX '19.

[4]  Suresh Jagannathan,et al.  CLOTHO: directed test generation for weakly consistent database systems , 2019, Proc. ACM Program. Lang..

[5]  Constantin Enea,et al.  On the complexity of checking transactional consistency , 2019, Proc. ACM Program. Lang..

[6]  Samer Al-Kiswany,et al.  An Analysis of Network-Partitioning Failures in Cloud Systems , 2018, OSDI.

[7]  Jonathan Lee,et al.  Proving the correct execution of concurrent services in zero-knowledge , 2018, IACR Cryptol. ePrint Arch..

[8]  Lorenzo Alvisi,et al.  Obladi: Oblivious Serializable Transactions in the Cloud , 2018, OSDI.

[9]  Peter Müller,et al.  Static serializability analysis for causal consistency , 2018, PLDI.

[10]  Suresh Jagannathan,et al.  Automated Detection of Serializability Violations under Weak Consistency , 2018, CONCUR.

[11]  David M. Eyers,et al.  LibSEAL: revealing service integrity violations using trusted execution , 2018, EuroSys.

[12]  Christof Fetzer,et al.  Pesos: policy enhanced secure object store , 2018, EuroSys.

[13]  Mihai Christodorescu,et al.  VeritasDB: High Throughput Key-Value Store with Integrity , 2018, IACR Cryptol. ePrint Arch..

[14]  Lingfan Yu,et al.  The Efficient Server Audit Problem, Deduplicated Re-execution, and the Web , 2017, SOSP.

[15]  Mikolás Janota,et al.  On the Quest for an Acyclic Graph , 2017, RCRA@AI*IA.

[16]  Lorenzo Alvisi,et al.  Seeing is Believing: A Client-Centric Specification of Database Isolation , 2017, PODC.

[17]  Jonathan Katz,et al.  vSQL: Verifying Arbitrary SQL Queries over Dynamic Outsourced Databases , 2017, 2017 IEEE Symposium on Security and Privacy (SP).

[18]  Hyeontaek Lim,et al.  Cicada: Dependably Fast Multi-Core In-Memory Transactions , 2017, SIGMOD Conference.

[19]  Ken Eguro,et al.  Concerto: A High Concurrency Key-Value Store with Integrity , 2017, SIGMOD Conference.

[20]  Peter Bailis,et al.  ACIDRain: Concurrency-Related Attacks on Database-Backed Web Applications , 2017, SIGMOD Conference.

[21]  Anurag Gupta,et al.  Amazon Aurora: Design Considerations for High Throughput Cloud-Native Relational Databases , 2017, SIGMOD Conference.

[22]  Chao Xie,et al.  Bringing Modular Concurrency Control to the Next Level , 2017, SIGMOD Conference.

[23]  Shweta Shinde,et al.  Panoply: Low-TCB Linux Applications With SGX Enclaves , 2017, NDSS.

[24]  Andrea C. Arpaci-Dusseau,et al.  Redundancy Does Not Imply Fault Tolerance: Analysis of Distributed Storage Reactions to Single Errors and Corruptions , 2017, FAST.

[25]  Peter Müller,et al.  Serializability for eventual consistency: criterion, analysis, and applications , 2017, POPL.

[26]  Tony Savor,et al.  Optimizing Space Amplification in RocksDB , 2017, CIDR.

[27]  Marijn J. H. Heule,et al.  SAT Competition 2016: Recent Developments , 2017, AAAI.

[28]  David M. Eyers,et al.  SCONE: Secure Linux Containers with Intel SGX , 2016, OSDI.

[29]  Emmett Witchel,et al.  Ryoan: A Distributed Sandbox for Untrusted Computation on Secret Data , 2016, OSDI.

[30]  Haibo Chen,et al.  Scaling Multicore Databases via Constrained Parallel Execution , 2016, SIGMOD Conference.

[31]  Srdjan Capkun,et al.  Verena: End-to-End Integrity Protection for Web Applications , 2016, 2016 IEEE Symposium on Security and Privacy (SP).

[32]  Krzysztof Czarnecki,et al.  Exponential Recency Weighted Average Branching Heuristic for SAT Solvers , 2016, AAAI.

[33]  Jonathan Katz,et al.  IntegriDB: Verifiable SQL for Outsourced Databases , 2015, CCS.

[34]  Arvind Krishnamurthy,et al.  Building consistent transactions with inconsistent replication , 2015, SOSP.

[35]  Sanjeev Kumar,et al.  Existential consistency: measuring and understanding consistency at Facebook , 2015, SOSP.

[36]  Chao Xie,et al.  High-performance ACID via modular concurrency control , 2015, SOSP.

[37]  David Lie,et al.  Caelus: Verifying the Consistency of Cloud Services with Battery-Powered Devices , 2015, 2015 IEEE Symposium on Security and Privacy.

[38]  Christos Gkantsidis,et al.  VC3: Trustworthy Data Analytics in the Cloud Using SGX , 2015, 2015 IEEE Symposium on Security and Privacy.

[39]  Alan J. Hu,et al.  SAT Modulo Monotonic Theories , 2014, AAAI.

[40]  Alexey Gotsman,et al.  A Framework for Transactional Consistency Models with Atomic Visibility , 2015, CONCUR.

[41]  Danfeng Zhang,et al.  Ironclad Apps: End-to-End Security via Automated Full-System Verification , 2014, OSDI.

[42]  Galen C. Hunt,et al.  Shielding Applications from an Untrusted Cloud with Haven , 2014, OSDI.

[43]  Mark Lillibridge,et al.  Torturing Databases for Fun and Profit , 2014, OSDI.

[44]  Yang Zhang,et al.  Extracting More Concurrency from Distributed Transactions , 2014, OSDI.

[45]  Martin Gebser,et al.  SAT Modulo Graphs: Acyclicity , 2014, JELIA.

[46]  Martin Gebser,et al.  Answer Set Programming as SAT modulo Acyclicity , 2014, ECAI.

[47]  Jie Wu,et al.  Consistency as a Service: Auditing Cloud Consistency , 2014, IEEE Transactions on Network and Service Management.

[48]  Indranil Gupta,et al.  Client-Centric Benchmarking of Eventual Consistency for Cloud Storage Systems , 2013, 2014 IEEE 34th International Conference on Distributed Computing Systems.

[49]  Bettina Kemme,et al.  Consistency anomalies in multi-tier architectures: automatic detection and prevention , 2013, The VLDB Journal.

[50]  Benjamin Braun,et al.  Verifying computations with state , 2013, IACR Cryptol. ePrint Arch..

[51]  Ali Ghodsi,et al.  Highly Available Transactions: Virtues and Limitations , 2013, Proc. VLDB Endow..

[52]  Divyakant Agrawal,et al.  Low-Latency Multi-Datacenter Databases using Replicated Commit , 2013, Proc. VLDB Endow..

[53]  Vyas Sekar,et al.  Towards verifiable resource accounting for outsourced computation , 2013, VEE '13.

[54]  Sunil Prabhakar,et al.  Trustworthy data from untrusted databases , 2013, 2013 IEEE 29th International Conference on Data Engineering (ICDE).

[55]  Tim Kraska,et al.  MDCC: multi-data center consistency , 2012, EuroSys '13.

[56]  Bettina Kemme,et al.  How consistent is your cloud application? , 2012, SoCC '12.

[57]  Christopher Frost,et al.  Spanner: Google's Globally-Distributed Database , 2012, OSDI.

[58]  Wojciech M. Golab,et al.  Toward a Principled Framework for Benchmarking Consistency , 2012, HotDep.

[59]  Dan R. K. Ports,et al.  Serializable Snapshot Isolation in PostgreSQL , 2012, Proc. VLDB Endow..

[60]  Sharad Malik,et al.  Predicting Serializability Violations: SMT-Based Search vs. DPOR-Based Search , 2011, Haifa Verification Conference.

[61]  Emin Gün Sirer,et al.  Logical attestation: an authorization architecture for trustworthy computing , 2011, SOSP.

[62]  Julian Dolby,et al.  Marathon: Detecting Atomic-Set Serializability Violations with Conflict Graphs , 2011, RV.

[63]  Adrian Perrig,et al.  Bootstrapping Trust in Modern Computers , 2011, Springer Briefs in Computer Science.

[64]  Helen J. Wang,et al.  Enabling Security in Cloud Storage SLAs with CloudProof , 2011, USENIX ATC.

[65]  Xiaozhou Li,et al.  Analyzing consistency properties for fun and profit , 2011, PODC '11.

[66]  Kevin Lee,et al.  Data Consistency Properties and the Trade-offs in Commercial Cloud Storage: the Consumers' Perspective , 2011, CIDR.

[67]  Idit Keidar,et al.  Venus: verification for untrusted cloud storage , 2010, CCSW '10.

[68]  Andreas Haeberlen,et al.  Accountable Virtual Machines , 2010, OSDI.

[69]  Xiaozhou Li,et al.  What Consistency Does Your Key-Value Store Actually Provide? , 2010, HotDep.

[70]  Adrian Perrig,et al.  TrustVisor: Efficient TCB Reduction and Attestation , 2010, 2010 IEEE Symposium on Security and Privacy.

[71]  Sharad Malik,et al.  Runtime checking of serializability in software transactional memory , 2010, 2010 IEEE International Symposium on Parallel & Distributed Processing (IPDPS).

[72]  Benjamin Livshits,et al.  Ripley: automatically securing web 2.0 applications through replicated execution , 2009, CCS.

[73]  Alan Fekete,et al.  Quantifying Isolation Anomalies , 2009, Proc. VLDB Endow..

[74]  Ronald L. Rivest,et al.  Introduction to Algorithms, third edition , 2009 .

[75]  Claude Castelluccia,et al.  Extending SAT Solvers to Cryptographic Problems , 2009, SAT.

[76]  Moni Naor,et al.  How Efficient Can Memory Checking Be? , 2009, TCC.

[77]  Cesare Tinelli,et al.  Handbook of Satisfiability , 2021, Handbook of Satisfiability.

[78]  Jonathan M. McCune,et al.  Efficient TCB Reduction and Attestation , 2009 .

[79]  Xiaozhou Li,et al.  Consistability: Describing Usually Consistent Systems , 2008, HotDep.

[80]  Frank Tip,et al.  Dynamic detection of atomic-set-serializability violations , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[81]  Michael K. Reiter,et al.  Flicker: an execution infrastructure for tcb minimization , 2008, Eurosys '08.

[82]  Nikolaj Bjørner,et al.  Z3: An Efficient SMT Solver , 2008, TACAS.

[83]  Eugene Goldberg,et al.  BerkMin: A Fast and Robust Sat-Solver , 2002, Discret. Appl. Math..

[84]  Feifei Li,et al.  Dynamic authenticated index structures for outsourced databases , 2006, SIGMOD Conference.

[85]  Min Xu,et al.  A serializability violation detector for shared-memory server programs , 2005, PLDI '05.

[86]  Trent Jaeger,et al.  Design and Implementation of a TCG-based Integrity Measurement Architecture , 2004, USENIX Security Symposium.

[87]  Niklas Sörensson,et al.  An Extensible SAT-solver , 2003, SAT.

[88]  David L. Dill,et al.  CVC: A Cooperating Validity Checker , 2002, CAV.

[89]  Gottfried Vossen,et al.  Transactional Information Systems: Theory, Algorithms, and the Practice of Concurrency Control and Recovery , 2002 .

[90]  C. Amza,et al.  Specification and implementation of dynamic Web site benchmarks , 2002, 2002 IEEE International Workshop on Workload Characterization.

[91]  M. Moskewicz,et al.  Chaff: engineering an efficient SAT solver , 2001, Proceedings of the 38th Design Automation Conference (IEEE Cat. No.01CH37232).

[92]  Miguel Oom Temudo de Castro,et al.  Practical Byzantine fault tolerance , 1999, OSDI '99.

[93]  Barbara Liskov,et al.  Weak Consistency: A Generalized Theory and Optimistic Implementations for Distributed Transactions , 1999 .

[94]  Marvin Theimer,et al.  Flexible update propagation for weakly consistent replication , 1997, SOSP.

[95]  Phillip B. Gibbons,et al.  Testing Shared Memories , 1997, SIAM J. Comput..

[96]  Jim Gray,et al.  A critique of ANSI SQL isolation levels , 1995, SIGMOD '95.

[97]  Jeannette M. Wing,et al.  Testing and Verifying Concurrent Objects , 1993, J. Parallel Distributed Comput..

[98]  Manuel Blum,et al.  Checking the correctness of memories , 1991, [1991] Proceedings 32nd Annual Symposium of Foundations of Computer Science.

[99]  Maurice Herlihy,et al.  Linearizability: a correctness condition for concurrent objects , 1990, TOPL.

[100]  Ralph C. Merkle,et al.  A Digital Signature Based on a Conventional Encryption Function , 1987, CRYPTO.

[101]  Philip A. Bernstein,et al.  Categories and Subject Descriptors: H.2.4 [Database Management]: Systems. , 2022 .

[102]  Marco A. Casanova,et al.  The Concurrency Control Problem for Database Systems , 1981, Lecture Notes in Computer Science.

[103]  Christos H. Papadimitriou,et al.  The serializability of concurrent database updates , 1979, JACM.

[104]  Leslie Lamport,et al.  How to Make a Multiprocessor Computer That Correctly Executes Multiprocess Programs , 2016, IEEE Transactions on Computers.

[105]  Philip A. Bernstein,et al.  Formal Aspects of Serializability in Database Concurrency Control , 1979, IEEE Transactions on Software Engineering.