Linear Temporal Logic Symbolic Model Checking

We are seeing an increased push in the use of formal verification techniques in safety-critical software and hardware in practice. Formal verification has been successfully used to verify systems such as air traffic control, airplane separation assurance, autopilot, CPU designs, life-support systems, medical equipment (such as devices which administer radiation), and many other systems which ensure human safety. This survey provides a perspective on the formal verification technique of linear temporal logic (LTL) symbolic model checking, from its history and evolution leading up to the state-of-the-art. We unify research from 1977 to 2009, providing a complete end-to-end analysis embracing a users' perspective by applying each step to a real-life aerospace example. We include an in-depth examination of the algorithms underlying the symbolic model-checking procedure, show proofs of important theorems, and point to directions of ongoing research. The primary focus is on model checking using LTL specifications, though other approaches are briefly discussed and compared to using LTL.

[1]  Johan Anthory Willem Kamp,et al.  Tense logic and the theory of linear order , 1968 .

[2]  Robert S. Streett,et al.  Propositional Dynamic Logic of Looping and Converse Is Elementarily Decidable , 1982, Inf. Control..

[3]  Alessio Lomuscio,et al.  Towards model checking interpreted systems , 2003, AAMAS '03.

[4]  Leslie Lamport,et al.  Proving Liveness Properties of Concurrent Programs , 1982, TOPL.

[5]  Randal E. Bryant,et al.  Symbolic Boolean manipulation with ordered binary-decision diagrams , 1992, CSUR.

[6]  Stephan Merz An Introduction to Model Checking , 2010 .

[7]  Saharon Shelah,et al.  On the temporal analysis of fairness , 1980, POPL '80.

[8]  E. Allen Emerson,et al.  Temporal and Modal Logic , 1991, Handbook of Theoretical Computer Science, Volume B: Formal Models and Sematics.

[9]  Joseph Y. Halpern,et al.  Decision procedures and expressiveness in the temporal logic of branching time , 1982, STOC '82.

[10]  A. Prasad Sistla,et al.  Safety, liveness and fairness in temporal logic , 1994, Formal Aspects of Computing.

[11]  Christel Baier,et al.  Principles of model checking , 2008 .

[12]  Moshe Y. Vardi,et al.  On ω-automata and temporal logic , 1989, STOC '89.

[13]  R. P. Kurshan,et al.  Automata-theoretic verification of coordinating processes , 1994 .

[14]  Leslie Lamport,et al.  Real-Time Model Checking Is Really Simple , 2005, CHARME.

[15]  Pierre Wolper,et al.  Simple on-the-fly automatic verification of linear temporal logic , 1995, PSTV.

[16]  Fausto Giunchiglia,et al.  NUSMV: a new symbolic model checker , 2000, International Journal on Software Tools for Technology Transfer.

[17]  Rod M. Burstall,et al.  Program Proving as Hand Simulation with a Little Induction , 1974, IFIP Congress.

[18]  Orna Kupferman,et al.  An Automata-Theoretic Approach to Reasoning about Infinite-State Systems , 2000, CAV.

[19]  Pierre Wolper,et al.  The Complementation Problem for Büchi Automata with Appplications to Temporal Logic , 1987, Theor. Comput. Sci..

[20]  Pierre Wolper,et al.  An Automata-Theoretic Approach to Automatic Program Verification (Preliminary Report) , 1986, LICS.

[21]  Marco Pistore,et al.  NuSMV 2: An OpenSource Tool for Symbolic Model Checking , 2002, CAV.

[22]  Fabio Somenzi,et al.  An Algorithm for Strongly Connected Component Analysis in n log n Symbolic Steps , 2006, Formal Methods Syst. Des..

[23]  Marsha Chechik,et al.  Model-checking infinite state-space systems with fine-grained abstractions using SPIN , 2001, SPIN '01.

[24]  Moshe Y. Vardi From Church and Prior to PSL , 2008, 25 Years of Model Checking.

[25]  Zohar Manna,et al.  The Temporal Logic of Reactive and Concurrent Systems , 1991, Springer New York.

[26]  Moshe Y. Vardi A temporal fixpoint calculus , 1988, POPL '88.

[27]  Kedar S. Namjoshi An Efficiently Checkable, Proof-Based Formulation of Vacuity in Model Checking , 2004, CAV.

[28]  Edmund M. Clarke,et al.  Efficient generation of counterexamples and witnesses in symbolic model checking , 1995, DAC '95.

[29]  Wang Yi,et al.  UPPAAL: Status & Developments , 1997, CAV.

[30]  Willem Visser,et al.  Model Checking Real Time Java Using Java PathFinder , 2005, ATVA.

[31]  Jaco Geldenhuys,et al.  Larger Automata and Less Work for LTL Model Checking , 2006, SPIN.

[32]  Robin Milner,et al.  An Algebraic Definition of Simulation Between Programs , 1971, IJCAI.

[33]  Steven P. Miller Will This Be Formal? , 2008, TPHOLs.

[34]  Lee S. Pike Real-Time System Verification by Kappa-Induction , 2013 .

[35]  David Aspinall,et al.  Formalising Java's Data Race Free Guarantee , 2007, TPHOLs.

[36]  Chin-Laung Lei,et al.  Efficient Model Checking in Fragments of the Propositional Mu-Calculus (Extended Abstract) , 1986, LICS.

[37]  Gerard J. Holzmann,et al.  The Model Checker SPIN , 1997, IEEE Trans. Software Eng..

[38]  Kavita Ravi,et al.  A Comparative Study of Symbolic Algorithms for the Computation of Fair Cycles , 2000, FMCAD.

[39]  Orna Kupferman,et al.  Sanity Checks in Formal Verification , 2006, CONCUR.

[40]  J. van Leeuwen,et al.  Theorem Proving in Higher Order Logics , 1999, Lecture Notes in Computer Science.

[41]  Chin-Laung Lei,et al.  Modalities for model checking (extended abstract): branching time strikes back , 1985, POPL.

[42]  John Havlicek,et al.  PSL AND SVA: TWO STANDARD ASSERTION LANGUAGES ADDRESSING COMPLEMENTARY ENGINEERING NEEDS , 2003 .

[43]  Pierre Wolper,et al.  An Automata-Theoretic Approach to Branching-Time Model Checking (Extended Abstract) , 1994, CAV.

[44]  Moshe Y. Vardi Sometimes and Not Never Re-revisited: On Branching Versus Linear Time , 1998, CONCUR.

[45]  Moshe Y. Vardi From Monadic Logic to PSL , 2008, Pillars of Computer Science.

[46]  E. Emerson,et al.  Modalities for model checking (extended abstract): branching time strikes back , 1985, ACM-SIGACT Symposium on Principles of Programming Languages.

[47]  Philippe Schnoebelen,et al.  Systems and Software Verification, Model-Checking Techniques and Tools , 2001 .

[48]  Amir Pnueli,et al.  Checking that finite state concurrent programs satisfy their linear specification , 1985, POPL.

[49]  C. Y. Lee Representation of switching circuits by binary-decision programs , 1959 .

[50]  Armin Biere,et al.  Bounded model checking , 2003, Adv. Comput..

[51]  A. Prasad Sistla,et al.  The complexity of propositional linear temporal logics , 1982, STOC '82.

[52]  Edmund M. Clarke,et al.  Symbolic Model Checking: 10^20 States and Beyond , 1990, Inf. Comput..

[53]  Fabio Somenzi,et al.  Vacuum Cleaning CTL Formulae , 2002, CAV.

[54]  Sheldon B. Akers,et al.  Binary Decision Diagrams , 1978, IEEE Transactions on Computers.

[55]  Joseph Sifakis,et al.  Specification and verification of concurrent systems in CESAR , 1982, Symposium on Programming.

[56]  Mats Per Erik Heimdahl,et al.  Proving the shalls , 2003, International Journal on Software Tools for Technology Transfer.

[57]  P. H. Lindsay Human Information Processing , 1977 .

[58]  Thierry Jéron,et al.  On-the-fly verification of finite transition systems , 1992, Formal Methods Syst. Des..

[59]  Sofia Cassel,et al.  Graph-Based Algorithms for Boolean Function Manipulation , 2012 .

[60]  Girish Keshav Palshikar Introduction to Model-checking , 2004 .

[61]  Fausto Giunchiglia,et al.  Improved Automata Generation for Linear Temporal Logic , 1999, CAV.

[63]  Leslie Lamport,et al.  Proving the Correctness of Multiprocess Programs , 1977, IEEE Transactions on Software Engineering.

[64]  Mats Per Erik Heimdahl,et al.  Model checking software requirement specifications using domain reduction abstraction , 2003, 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings..

[65]  Moshe Y. Vardi Automata-theoretic techniques for temporal reasoning , 2007, Handbook of Modal Logic.

[66]  C. Eisner,et al.  Efficient Detection of Vacuity in ACTL Formulaas , 1997, CAV.

[67]  Bowen Alpern,et al.  Defining Liveness , 1984, Inf. Process. Lett..

[68]  S. Safra,et al.  On the complexity of omega -automata , 1988, [Proceedings 1988] 29th Annual Symposium on Foundations of Computer Science.

[69]  Orna Kupferman,et al.  Weak alternating automata are not that weak , 1997, Proceedings of the Fifth Israeli Symposium on Theory of Computing and Systems.

[70]  Pierre Wolper,et al.  Reasoning about infinite computation paths , 1983, 24th Annual Symposium on Foundations of Computer Science (sfcs 1983).

[71]  Joanne M. Atlee,et al.  Feasibility of model checking software requirements: a case study , 1996, Proceedings of 11th Annual Conference on Computer Assurance. COMPASS '96.

[72]  Ilan Beer,et al.  Efficient Detection of Vacuity in Temporal Model Checking , 2001, Formal Methods Syst. Des..

[73]  Denis Poitrenaud,et al.  SPOT: an extensible model checking library using transition-based generalized Bu/spl uml/chi automata , 2004, The IEEE Computer Society's 12th Annual International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunications Systems, 2004. (MASCOTS 2004). Proceedings..

[74]  Clifford Stein,et al.  Introduction to Algorithms, 2nd edition. , 2001 .

[75]  Avner Landver,et al.  The ForSpec Temporal Logic: A New Temporal Property-Specification Language , 2002, TACAS.

[76]  Fabio Somenzi,et al.  Efficient Büchi Automata from LTL Formulae , 2000, CAV.

[77]  E. Kindler Safety and Liveness Properties: A Survey , 2007 .

[78]  Leslie Lamport,et al.  Distributed Systems: Methods and Tools for Specification, An Advanced Course, April 3-12, 1984 and April 16-25, 1985, Munich, Germany , 1985, Advanced Course: Distributed Systems.

[79]  Valentin Goranko,et al.  Logic in Computer Science: Modelling and Reasoning About Systems , 2007, J. Log. Lang. Inf..

[80]  Doron A. Peled,et al.  All from One, One for All: on Model Checking Using Representatives , 1993, CAV.

[81]  Fausto Giunchiglia,et al.  Planning via Model Checking: A Decision Procedure for AR , 1997, ECP.

[82]  Sandeep K. Shukla,et al.  A New Heuristic for Bad Cycle Detection Using BDDs , 1997, Formal Methods Syst. Des..

[83]  Dana Fisman,et al.  The Temporal Logic Sugar , 2001, CAV.

[84]  李幼升,et al.  Ph , 1989 .

[85]  Moshe Y. Vardi Branching vs. Linear Time: Final Showdown , 2001, TACAS.

[86]  Frank Wolter,et al.  Handbook of Modal Logic , 2007, Studies in logic and practical reasoning.

[87]  Jean-Michel Couvreur,et al.  On-the-Fly Verification of Linear Temporal Logic , 1999, World Congress on Formal Methods.

[88]  Jan van Leeuwen,et al.  Handbook Of Theoretical Computer Science, Vol. A , 1990 .

[89]  Detlef Sieling The Nonapproximability of OBDD Minimization , 2002, Inf. Comput..

[90]  E. Allen Emerson,et al.  The Complexity of Tree Automata and Logics of Programs , 1999, SIAM J. Comput..

[91]  James F. Allen Towards a General Theory of Action and Time , 1984, Artif. Intell..

[92]  Pierre Wolper,et al.  The Complementation Problem for Büchi Automata with Applications to Temporal Logic (Extended Abstract) , 1985, ICALP.

[93]  Edmund M. Clarke,et al.  Another Look at LTL Model Checking , 1994, CAV.

[94]  Dimitra Giannakopoulou,et al.  From States to Transitions: Improving Translation of LTL Formulae to Büchi Automata , 2002, FORTE.

[95]  Pierre Wolper Temporal Logic Can Be More Expressive , 1983, Inf. Control..

[96]  Moshe Y. Vardi Linear vs. branching time: a complexity-theoretic perspective , 1998, Proceedings. Thirteenth Annual IEEE Symposium on Logic in Computer Science (Cat. No.98CB36226).

[97]  Shoham Ben-David,et al.  Fair Cycle Detection using Description Logic Reasoning , 2009, Description Logics.

[98]  Thomas A. Henzinger,et al.  HYTECH: a model checker for hybrid systems , 1997, International Journal on Software Tools for Technology Transfer.

[99]  Nagisa Ishiura,et al.  Shared binary decision diagram with attributed edges for efficient Boolean function manipulation , 1990, 27th ACM/IEEE Design Automation Conference.

[100]  Victoria Allen,et al.  All for one. , 2013, Journal of obstetrics and gynaecology Canada : JOGC = Journal d'obstetrique et gynecologie du Canada : JOGC.

[101]  Javier Esparza,et al.  Decidability of model checking for infinite-state concurrent systems , 1997, Acta Informatica.

[102]  Nissim Francez,et al.  Generalized fair termination , 1984, POPL '84.

[103]  R. K. Shyamasundar,et al.  Introduction to algorithms , 1996 .

[104]  Fred Kröger,et al.  Temporal Logic of Programs , 1987, EATCS Monographs on Theoretical Computer Science.

[105]  Orna Kupferman,et al.  Vacuity Detection in Temporal Model Checking , 1999, CHARME.

[106]  Erik Meineche Schmidt,et al.  The Complexity of Equivalence and Containment for Free Single Variable Program Schemes , 1978, ICALP.

[107]  Cliff B. Jones,et al.  Specification and Design of (Parallel) Programs , 1983, IFIP Congress.

[108]  Viktor Schuppan,et al.  Liveness Checking as Safety Checking for Infinite State Spaces , 2006, INFINITY.

[109]  G. B. Finelli,et al.  The Infeasibility of Quantifying the Reliability of Life-Critical Real-Time Software , 1993, IEEE Trans. Software Eng..

[110]  Peter A. Beerel,et al.  Implicit enumeration of strongly connected components and anapplication to formal verification , 2000, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[111]  Frank Wolter,et al.  Handbook of Modal Logic, Volume 3 (Studies in Logic and Practical Reasoning) , 2006 .

[112]  Grigore Rosu,et al.  Allen Linear (Interval) Temporal Logic - Translation to LTL and Monitor Synthesis , 2006, CAV.

[113]  Xavier Thirioux Simple and Efficient Translation from LTL Formulas to Buchi Automata , 2002, Electron. Notes Theor. Comput. Sci..

[114]  Marsha Chechik,et al.  Extending Extended Vacuity , 2004, FMCAD.

[115]  Kathi Fisler,et al.  Is There a Best Symbolic Cycle-Detection Algorithm? , 2001, TACAS.

[116]  Robert S. Streett Propositional Dynamic Logic of looping and converse , 1981, STOC '81.

[117]  Mihalis Yannakakis,et al.  On nested depth first search , 1996, The Spin Verification System.

[118]  Pierre Wolper,et al.  An automata-theoretic approach to branching-time model checking , 2000, JACM.

[119]  Moshe Y. Vardi Automata-Theoretic Model Checking Revisited , 2007, VMCAI.

[120]  Robert K. Brayton,et al.  BDD-Based Debugging Of Design Using Language Containment and Fair CTL , 1993, CAV.

[121]  Bowen Alpern,et al.  Recognizing safety and liveness , 2005, Distributed Computing.

[122]  Mordechai Ben-Ari,et al.  The temporal logic of branching time , 1981, POPL '81.

[123]  Roberto Sebastiani,et al.  "More Deterministic" vs. "Smaller" Büchi Automata for Efficient LTL Model Checking , 2003, CHARME.

[124]  Peter Linz,et al.  An Introduction to Formal Languages and Automata , 1997 .

[125]  Fabio Somenzi,et al.  An Algorithm for Strongly Connected Component Analysis in n log n Symbolic Steps , 2000, Formal Methods Syst. Des..

[126]  A. Prasad Sistla,et al.  Deciding branching time logic , 1984, STOC '84.

[127]  Edmund M. Clarke,et al.  The Birth of Model Checking , 2008, 25 Years of Model Checking.

[128]  Larry J. Stockmeyer,et al.  Improved upper and lower bounds for modal logics of programs , 1985, STOC '85.

[129]  Moshe Y. Vardi An Automata-Theoretic Approach to Linear Temporal Logic , 1996, Banff Higher Order Workshop.

[130]  Dimitra Giannakopoulou,et al.  Model checking for concurrent software architectures , 1999 .

[131]  Orna Kupferman,et al.  Model Checking of Safety Properties , 1999, CAV.

[132]  Paul Gastin,et al.  Fast LTL to Büchi Automata Translation , 2001, CAV.

[133]  Edmund M. Clarke,et al.  Verification Tools for Finite-State Concurrent Systems , 1993, REX School/Symposium.

[134]  Howard Barringer,et al.  Temporal Logic with Fixed Points , 1987, Temporal Logic in Specification.

[135]  Carsten Fritz,et al.  Concepts of Automata Construction from LTL , 2005, LPAR.

[136]  Edmund M. Clarke,et al.  Symbolic model checking for sequential circuit verification , 1993, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[137]  Edmund M. Clarke,et al.  Design and Synthesis of Synchronization Skeletons Using Branching-Time Temporal Logic , 1981, Logic of Programs.

[138]  Moshe Y. Vardi,et al.  Büchi Complementation and Size-Change Termination , 2009, TACAS.

[139]  Victor Carreño,et al.  Formal Analysis of the Operational Concept for the Small Aircraft Transportation System , 2006, RODIN Book.

[140]  Robert K. Brayton,et al.  Efficient omega-Regular Language Containment , 1992, CAV.

[141]  Orna Grumberg,et al.  Regular Vacuity , 2005, CHARME.

[142]  Marsha Chechik,et al.  How Vacuous Is Vacuous? , 2004, TACAS.

[143]  Heinz Erzberger,et al.  The automated airspace concept , 2001 .

[144]  H. Andersen An Introduction to Binary Decision Diagrams , 1997 .

[145]  Ingo Wegener,et al.  Reduction of OBDDs in Linear Time , 1993, Inf. Process. Lett..

[146]  Lee Pike Real-Time System Verification by k-Induction , 2005 .

[147]  Pierre Wolper,et al.  Memory-efficient algorithms for the verification of temporal properties , 1990, Formal Methods Syst. Des..

[148]  David Notkin,et al.  Model checking large software specifications , 1996, SIGSOFT '96.

[149]  Antti Valmari,et al.  A stubborn attack on state explosion , 1990, Formal Methods Syst. Des..

[150]  Moshe Y. Vardi,et al.  LTL Satisfiability Checking , 2007, SPIN.

[151]  Peter A. Beerel,et al.  Implicit enumeration of strongly connected components , 1999, 1999 IEEE/ACM International Conference on Computer-Aided Design. Digest of Technical Papers (Cat. No.99CH37051).

[152]  Jean-François Raskin,et al.  Improved Algorithms for the Automata-Based Approach to Model-Checking , 2007, TACAS.

[153]  Joseph Y. Halpern,et al.  “Sometimes” and “not never” revisited: on branching versus linear time temporal logic , 1986, JACM.

[154]  Kenneth L. McMillan,et al.  Symbolic model checking: an approach to the state explosion problem , 1992 .

[155]  J. R. Büchi On a Decision Method in Restricted Second Order Arithmetic , 1990 .

[156]  E. Allen Emerson,et al.  Generalized Quantitative Temporal Reasoning: An Automata Theoretic Approach , 1997, TAPSOFT.

[157]  Richard E. Ladner,et al.  Propositional Dynamic Logic of Regular Programs , 1979, J. Comput. Syst. Sci..

[158]  James R. Larus,et al.  Debugging temporal specifications with concept analysis , 2003, PLDI '03.

[159]  Fred Kröger LAR: A logic of algorithmic reasoning , 2004, Acta Informatica.

[160]  Armin Biere,et al.  Symbolic Model Checking without BDDs , 1999, TACAS.

[161]  Amir Pnueli,et al.  Algorithmic Verification of Linear Temporal Logic Specifications , 1998, ICALP.

[162]  Marsha Chechik,et al.  Model checking with multi-valued temporal logics , 2001, Proceedings 31st IEEE International Symposium on Multiple-Valued Logic.

[163]  Christel Baier,et al.  Principles of Model Checking (Representation and Mind Series) , 2008 .

[164]  S. P. Miller,et al.  Software safety analysis of a flight management system vertical navigation function - a status report , 2003, Digital Avionics Systems Conference, 2003. DASC '03. The 22nd.

[165]  Peter Linz An introduction to formal languages and automata (2nd ed.) , 1996 .

[166]  Chin-Laung Lei,et al.  Modalities for Model Checking: Branching Time Logic Strikes Back , 1987, Sci. Comput. Program..

[167]  Tevfik Bultan,et al.  Eliminating synchronization faults in air traffic control software via design for verification with concurrency controllers , 2007, Automated Software Engineering.

[168]  Edmund M. Clarke,et al.  Model Checking , 1999, Handbook of Automated Reasoning.

[169]  Pierre Wolper,et al.  Reasoning About Infinite Computations , 1994, Inf. Comput..

[170]  Enrico Tronci,et al.  Model-Checking Based on Fluid Petri Nets for the Temperature Control System of the ICARO Co-generative Plant , 2002, SAFECOMP.

[171]  Orna Grumberg,et al.  Enhanced Vacuity Detection in Linear Temporal Logic , 2003, CAV.

[172]  Enrico Macii,et al.  Markovian analysis of large finite state machines , 1996, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[173]  Howard Barringer,et al.  Practical CTL* model checking: Should SPIN be extended? , 2000, International Journal on Software Tools for Technology Transfer.

[174]  Tiziano Villa,et al.  VIS: A System for Verification and Synthesis , 1996, CAV.

[175]  Limor Fix,et al.  Fifteen Years of Formal Property Verification in Intel , 2008, 25 Years of Model Checking.

[176]  Srikanth Vijayaraghavan,et al.  A Practical Guide for SystemVerilog Assertions , 2005 .

[177]  Leslie Lamport,et al.  "Sometime" is sometimes "not never": on the temporal logic of programs , 1980, POPL '80.