Probabilistic Program Analysis with Martingales

We present techniques for the analysis of infinite state probabilistic programs to synthesize probabilistic invariants and prove almost-sure termination. Our analysis is based on the notion of (super) martingales from probability theory. First, we define the concept of (super) martingales for loops in probabilistic programs. Next, we present the use of concentration of measure inequalities to bound the values of martingales with high probability. This directly allows us to infer probabilistic bounds on assertions involving the program variables. Next, we present the notion of a super martingale ranking function (SMRF) to prove almost sure termination of probabilistic programs. Finally, we extend constraint-based techniques to synthesize martingales and super-martingale ranking functions for probabilistic programs. We present some applications of our approach to reason about invariance and termination of small but complex probabilistic programs.

[1]  Patrick Cousot,et al.  Probabilistic Abstract Interpretation , 2012, ESOP.

[2]  Annabelle McIver,et al.  Abstraction, Refinement And Proof For Probabilistic Systems (Monographs in Computer Science) , 2004 .

[3]  Axel Legay,et al.  Statistical Model Checking of Mixed-Analog Circuits with an Application to a Third Order Delta-Sigma Modulator , 2009, Haifa Verification Conference.

[4]  Henny B. Sipma,et al.  Termination of Polynomial Programs , 2005, VMCAI.

[5]  Javier Esparza,et al.  Proving Termination of Probabilistic Programs Using Patterns , 2012, CAV.

[6]  Henny B. Sipma,et al.  Non-linear loop invariant generation using Gröbner bases , 2004, POPL.

[7]  Marta Z. Kwiatkowska,et al.  PRISM: probabilistic model checking for performance and reliability analysis , 2009, PERV.

[8]  Andreas Podelski,et al.  A Complete Method for the Synthesis of Linear Ranking Functions , 2004, VMCAI.

[9]  Richard Lassaigne,et al.  Probabilistic Verification and Approximation , 2006, Electron. Notes Theor. Comput. Sci..

[10]  Russ Bubley,et al.  Randomized algorithms , 1995, CSUR.

[11]  Lijun Zhang,et al.  Probabilistic CEGAR , 2008, CAV.

[12]  Olivier Bournez,et al.  Proving Positive Almost-Sure Termination , 2005, RTA.

[13]  André Platzer,et al.  Differential Dynamic Logic for Hybrid Systems , 2008, Journal of Automated Reasoning.

[14]  Roberto Bagnara,et al.  The Parma Polyhedra Library: Toward a complete set of numerical abstractions for the analysis and verification of hardware and software systems , 2006, Sci. Comput. Program..

[15]  David Monniaux,et al.  Abstract interpretation of programs as Markov decision processes , 2003, Sci. Comput. Program..

[16]  Patrick Cousot,et al.  Proving Program Invariance and Termination by Parametric Abstraction, Lagrangian Relaxation and Semidefinite Programming , 2005, VMCAI.

[17]  Alessandro Panconesi,et al.  Concentration of Measure for the Analysis of Randomized Algorithms , 2009 .

[18]  Nicolas Halbwachs,et al.  Automatic discovery of linear restraints among variables of a program , 1978, POPL.

[19]  Annabelle McIver,et al.  Linear-Invariant Generation for Probabilistic Programs: - Automated Support for Proof-Based Methods , 2010, SAS.

[20]  Kousha Etessami,et al.  Analysis of Recursive Game Graphs Using Data Flow Equations , 2004, VMCAI.

[21]  Sumit Gulwani,et al.  Program analysis as constraint solving , 2008, PLDI '08.

[22]  André Platzer,et al.  Stochastic Differential Dynamic Logic for Stochastic Hybrid Programs , 2011, CADE.

[23]  Harald Ganzinger,et al.  Programs as Data Objects , 1986, Lecture Notes in Computer Science.

[24]  Antoine Miné,et al.  A New Numerical Abstract Domain Based on Difference-Bound Matrices , 2001, PADO.

[25]  Philip Wadler.  Call-by-Value Is Dual to Call-by-Name - Reloaded , 2005, RTA.

[26]  Annabelle McIver,et al.  Abstraction, Refinement and Proof for Probabilistic Systems , 2004, Monographs in Computer Science.

[27]  Nikolaj Bjørner,et al.  Automated Deduction - CADE-23 - 23rd International Conference on Automated Deduction, Wroclaw, Poland, July 31 - August 5, 2011. Proceedings , 2011, CADE.

[28]  Andreas Podelski,et al.  Transition invariants , 2004, Proceedings of the 19th Annual IEEE Symposium on Logic in Computer Science, 2004..

[29]  Christel Baier,et al.  Symbolic Model Checking for Probabilistic Processes , 1997, ICALP.

[30]  Helmut Seidl,et al.  Precise interprocedural analysis through linear algebra , 2004, POPL.

[31]  Jean Goubault-Larrecq,et al.  A generalization of p-boxes to affine arithmetic , 2011, Computing.

[32]  F. G. Foster.  On the Stochastic Matrices Associated with Certain Queuing Processes , 1953 .

[33]  Henny B. Sipma,et al.  Synthesis of Linear Ranking Functions , 2001, TACAS.

[34]  Annabelle McIver,et al.  Developing and Reasoning About Probabilistic Programs in pGCL , 2004, PSSE.

[35]  Håkan L. S. Younes,et al.  Statistical probabilistic model checking with a focus on time-bounded properties , 2006, Inf. Comput..

[36]  Enric Rodríguez-Carbonell,et al.  Automatic Generation of Polynomial Loop Invariants: Algebraic Foundations , 2004, ISSAC '04.

[37]  Rajeev Alur,et al.  A Temporal Logic of Nested Calls and Returns , 2004, TACAS.

[38]  Henny B. Sipma,et al.  Linear Invariant Generation Using Non-linear Constraint Solving , 2003, CAV.

[39]  Augusto Sampaio,et al.  Refinement Techniques in Software Engineering, First Pernambuco Summer School on Software Engineering, PSSE 2004, Recife, Brazil, November 23-December 5, 2004, Revised Lectures , 2006, PSSE.