Termination Analysis of Probabilistic Programs Through Positivstellensatz's

We consider nondeterministic probabilistic programs with the most basic liveness property of termination. We present efficient methods for termination analysis of nondeterministic probabilistic programs with polynomial guards and assignments. Our approach is through synthesis of polynomial ranking supermartingales, that on one hand significantly generalizes linear ranking supermartingales and on the other hand is a counterpart of polynomial ranking-functions for proving termination of nonprobabilistic programs. The approach synthesizes polynomial ranking-supermartingales through Positivstellensatz’s, yielding an efficient method which is not only sound, but also semi-complete over a large subclass of programs. We show experimental results to demonstrate that our approach can handle several classical programs with complex polynomial guards and assignments, and can synthesize efficient quadratic ranking-supermartingales when a linear one does not exist even for simple affine programs.

[1]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[2]  Robert W. Floyd,et al.  Assigning Meanings to Programs , 1993 .

[3]  Holger Hermanns,et al.  Probabilistic Termination , 2015, POPL.

[4]  Krishnendu Chatterjee,et al.  Algorithmic analysis of qualitative and quantitative termination problems for affine probabilistic programs , 2015, POPL.

[5]  Allen Van Gelder,et al.  Termination detection in logic programs using argument sizes (extended abstract) , 1991, PODS.

[6]  J. G. Pierce,et al.  Geometric Algorithms and Combinatorial Optimization , 2016 .

[7]  Patrick Cousot,et al.  Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints , 1977, POPL.

[8]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[9]  Andreas Podelski,et al.  A Complete Method for the Synthesis of Linear Ranking Functions , 2004, VMCAI.

[10]  Annabelle McIver,et al.  Developing and Reasoning About Probabilistic Programs in pGCL , 2004, PSSE.

[11]  H. P. Williams.  THEORY OF LINEAR AND INTEGER PROGRAMMING (Wiley-Interscience Series in Discrete Mathematics and Optimization) , 1989 .

[12]  Henny B. Sipma,et al.  Linear Ranking with Reachability , 2005, CAV.

[13]  Sriram Sankaranarayanan,et al.  Probabilistic Program Analysis with Martingales , 2013, CAV.

[14]  Marta Z. Kwiatkowska,et al.  PRISM 4.0: Verification of Probabilistic Real-Time Systems , 2011, CAV.

[15]  Henny B. Sipma,et al.  Termination of Polynomial Programs , 2005, VMCAI.

[16]  Alessandro Panconesi,et al.  Concentration of Measure for the Analysis of Randomized Algorithms , 2009 .

[17]  Annabelle McIver,et al.  Abstraction, Refinement And Proof For Probabilistic Systems (Monographs in Computer Science) , 2004 .

[18]  Russ Bubley,et al.  Randomized algorithms , 1995, CSUR.

[19]  David Monniaux.  An Abstract Analysis of the Probabilistic Termination of Programs , 2001, SAS.

[20]  R. Tennant.  Algebra , 1941, Nature.

[21]  K. Schmüdgen.  TheK-moment problem for compact semi-algebraic sets , 1991 .

[22]  Zhenbing Zeng,et al.  Generating exact nonlinear ranking functions by symbolic-numeric hybrid method , 2013, J. Syst. Sci. Complex..

[23]  Chaochen Zhou,et al.  Recent advances in program verification through computer algebra , 2009, Frontiers of Computer Science in China.

[24]  Sumit Gulwani,et al.  Static analysis for probabilistic programs: inferring whole program properties from finitely many paths , 2013, PLDI.

[25]  Christel Baier,et al.  Principles of model checking , 2008 .

[26]  S. Sullivant,et al.  Emerging applications of algebraic geometry , 2009 .

[27]  Azaria Paz,et al.  Introduction to probabilistic automata (Computer science and applied mathematics) , 1971 .

[28]  J. Kemeny,et al.  Denumerable Markov chains , 1969 .

[29]  Hadas Kress-Gazit,et al.  Temporal-Logic-Based Reactive Mission and Motion Planning , 2009, IEEE Transactions on Robotics.

[30]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[31]  K. Schmüdgen.  TheK-moment problem for compact semi-algebraic sets , 1991 .

[32]  Henny B. Sipma,et al.  Synthesis of Linear Ranking Functions , 2001, TACAS.

[33]  Radhia Cousot.  Verification, Model Checking, and Abstract Interpretation, 6th International Conference, VMCAI 2005, Paris, France, January 17-19, 2005, Proceedings , 2005, VMCAI.

[34]  Alexander Schrijver,et al.  Theory of linear and integer programming , 1986, Wiley-Interscience series in discrete mathematics and optimization.

[35]  W. Hoeffding.  Probability Inequalities for sums of Bounded Random Variables , 1963 .

[36]  Christel Baier,et al.  Probabilistic ω-automata , 2012, JACM.

[37]  David Williams,et al.  Probability with Martingales , 1991, Cambridge mathematical textbooks.

[38]  D. Handelman.  Representing polynomials by positive linear functions on compact convex polyhedra. , 1988 .

[39]  ChatterjeeKrishnendu,et al.  Algorithmic Analysis of Qualitative and Quantitative Termination Problems for Affine Probabilistic Programs , 2016 .

[40]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[41]  Javier Esparza,et al.  Proving Termination of Probabilistic Programs Using Patterns , 2012, CAV.

[42]  J. Norris.  Appendix: probability and measure , 1997 .

[43]  Marsha Chechik,et al.  Tools and Algorithms for the Construction and Analysis of Systems , 2016, Lecture Notes in Computer Science.

[44]  Claus Scheiderer,et al.  Positivity and sums of squares: A guide to recent results , 2009 .

[45]  D. S. Arnon,et al.  Algorithms in real algebraic geometry , 1988 .

[46]  Patrick Cousot,et al.  Proving Program Invariance and Termination by Parametric Abstraction, Lagrangian Relaxation and Semidefinite Programming , 2005, VMCAI.

[47]  A. Tarski.  A Decision Method for Elementary Algebra and Geometry , 2023 .

[48]  Azaria Paz,et al.  Probabilistic automata , 2003 .

[49]  Annabelle McIver,et al.  Abstraction, Refinement and Proof for Probabilistic Systems , 2004, Monographs in Computer Science.

[50]  Tiziana Margaria,et al.  Tools and algorithms for the construction and analysis of systems: a special issue for TACAS 2017 , 2001, International Journal on Software Tools for Technology Transfer.

[51]  Alan J. Hu,et al.  Proving termination of nonlinear command sequences , 2013, Formal Aspects of Computing.

[52]  Olivier Bournez,et al.  Proving Positive Almost-Sure Termination , 2005, RTA.

[53]  F. G. Foster.  On the Stochastic Matrices Associated with Certain Queuing Processes , 1953 .

[54]  Jürgen Giesl.  Term Rewriting and Applications, 16th International Conference, RTA 2005, Nara, Japan, April 19-21, 2005, Proceedings , 2005, RTA.