Verification of Threshold-Based Distributed Algorithms by Decomposition to Decidable Logics

Verification of fault-tolerant distributed protocols is an immensely difficult task. Often, in these protocols, thresholds on set cardinalities are used both in the process code and in its correctness proof, e.g., a process can perform an action only if it has received an acknowledgment from at least half of its peers. Verification of threshold-based protocols is extremely challenging as it involves two kinds of reasoning: first-order reasoning about the unbounded state of the protocol, together with reasoning about sets and cardinalities. In this work, we develop a new methodology for decomposing the verification task of such protocols into two decidable logics: EPR and BAPA. Our key insight is that such protocols use thresholds in a restricted way as a means to obtain certain properties of “intersection” between sets. We define a language for expressing such properties, and present two translations: to EPR and BAPA. The EPR translation allows verifying the protocol while assuming these properties, and the BAPA translation allows verifying the correctness of the properties. We further develop an algorithm for automatically generating the properties needed for verifying a given protocol, facilitating fully automated deductive verification. Using this technique we have verified several challenging protocols, including Byzantine one-step consensus, hybrid reliable broadcast and fast Byzantine Paxos.

[1]  Deian Stefan,et al.  Pretend synchrony: synchronous verification of asynchronous distributed programs , 2019, Proc. ACM Program. Lang..

[2]  Jean-Philippe Martin,et al.  Fast Byzantine Consensus , 2006, IEEE Transactions on Dependable and Secure Computing.

[3]  Shmuel Sagiv,et al.  Paxos made EPR: decidable reasoning about distributed protocols , 2017, Proc. ACM Program. Lang..

[4]  Ramakrishna Kotla,et al.  Zyzzyva: speculative byzantine fault tolerance , 2007, TOCS.

[5]  Thomas A. Henzinger,et al.  A Logic-Based Framework for Verifying Consensus Algorithms , 2014, VMCAI.

[6]  Thomas A. Henzinger,et al.  PSync: a partially synchronous language for fault-tolerant distributed algorithms , 2016, POPL.

[7]  Nikolaj Bjørner,et al.  Z3: An Efficient SMT Solver , 2008, TACAS.

[8]  Leonardo Mendonça de Moura,et al.  Complete Instantiation for Quantified Formulas in Satisfiabiliby Modulo Theories , 2009, CAV.

[9]  Harry R. Lewis,et al.  Complexity Results for Classes of Quantificational Formulas , 1980, J. Comput. Syst. Sci..

[10]  Xi Wang,et al.  Verdi: a framework for implementing and formally verifying distributed systems , 2015, PLDI.

[11]  Joao Marques-Silva,et al.  Fast, flexible MUS enumeration , 2015, Constraints.

[12]  Lorenzo Alvisi,et al.  Fast Byzantine Paxos , 2004 .

[13]  Robbert van Renesse,et al.  Bosco: One-Step Byzantine Asynchronous Consensus , 2008, DISC.

[14]  Paulo Veríssimo,et al.  Velisarios: Byzantine Fault-Tolerant Protocols Powered by Coq , 2018, ESOP.

[15]  Nathalie Bertrand,et al.  Verification of Randomized Distributed Algorithms under Round-Rigid Adversaries , 2018 .

[16]  Jochen Hoenicke,et al.  Reducing liveness to safety in first-order logic , 2017, Proc. ACM Program. Lang..

[17]  Helmut Veith,et al.  A short counterexample property for safety and liveness verification of fault-tolerant distributed algorithms , 2016, POPL.

[18]  Leslie Lamport,et al.  The part-time parliament , 1998, TOCS.

[19]  Ulrich Schmid,et al.  Booting clock synchronization in partially synchronous systems with hybrid process and link failures , 2007, Distributed Computing.

[20]  Florian Zuleger,et al.  Verifying safety of synchronous fault-tolerant algorithms by bounded model checking , 2019, International Journal on Software Tools for Technology Transfer.

[21]  Igor Konnov,et al.  Reachability in Parameterized Systems: All Flavors of Threshold Automata , 2018, CONCUR.

[22]  Nikolaj Bjørner,et al.  Deciding Effectively Propositional Logic Using DPLL and Substitution Sets , 2008, IJCAR.

[23]  Helmut Veith,et al.  On the completeness of bounded model checking for threshold-based distributed algorithms: Reachability , 2014, Inf. Comput..

[24]  Leslie Lamport,et al.  Lower bounds for asynchronous consensus , 2006, Distributed Computing.

[25]  Ruzica Piskac,et al.  Decision Procedures for Program Synthesis and Verification , 2011 .

[26]  Srinath T. V. Setty,et al.  IronFleet: proving practical distributed systems correct , 2015, SOSP.

[27]  Igor Konnov,et al.  Synthesis of Distributed Algorithms with Parameterized Threshold Guards , 2018, OPODIS.

[28]  Ilya Sergey,et al.  Programming and proving with distributed protocols , 2017, Proc. ACM Program. Lang..

[29]  Igor Konnov,et al.  ByMC: Byzantine Model Checker , 2018, ISoLA.

[30]  Viktor Kuncak,et al.  An Algorithm for Deciding BAPA: Boolean Algebra with Presburger Arithmetic , 2005, CADE.

[31]  Yanhong A. Liu,et al.  From clarity to efficiency for distributed algorithms , 2012, OOPSLA '12.

[32]  Kenneth L. McMillan,et al.  Deductive Verification in Decidable Fragments with Ivy , 2018, SAS.

[33]  Helmut Veith,et al.  Para\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^2$$\end{document}2: parameterized path reduction, acceleration, a , 2017, Formal Methods in System Design.

[34]  Jorge A. Navas,et al.  Verification of Fault-Tolerant Protocols with Sally , 2018, NFM.

[35]  Mark Bickford,et al.  Formal Specification, Verification, and Implementation of Fault-Tolerant Systems using EventML , 2015, Electron. Commun. Eur. Assoc. Softw. Sci. Technol..

[36]  Sam Toueg,et al.  Simulating authenticated broadcasts to derive simple fault-tolerant algorithms , 1987, Distributed Computing.

[37]  Cesare Tinelli,et al.  A New Decision Procedure for Finite Sets and Cardinality Constraints in SMT , 2016, IJCAR.

[38]  Michael D. Ernst,et al.  Planning for change in a formal verification of the raft consensus protocol , 2016, CPP.

[39]  Nikolaj Bjørner,et al.  Cardinalities and universal quantifiers for verifying parameterized systems , 2016, PLDI.

[40]  Jochen Hoenicke,et al.  Temporal Prophecy for Proving Temporal Properties of Infinite-State Systems , 2018, 2018 Formal Methods in Computer Aided Design (FMCAD).

[41]  Kenneth L. McMillan,et al.  Ivy: safety verification by interactive generalization , 2016, PLDI.

[42]  Kenneth L. McMillan,et al.  Modularity for decidability of deductive verification with applications to distributed systems , 2018, PLDI.

[43]  Ramakrishna Kotla,et al.  Revisiting Fast Practical Byzantine Fault Tolerance , 2017, ArXiv.