Exploiting Robust Optimization for Interval Probabilistic Bisimulation

Verification of PCTL properties of MDPs with convex uncertainties has been investigated recently by Puggelli et al. However, model checking algorithms typically suffer from the state space explosion problem. In this paper, we discuss the use of probabilistic bisimulation to reduce the size of such an MDP while preserving the PCTL properties it satisfies. As a core part, we show that deciding bisimilarity of a pair of states can be encoded as adjustable robust counterpart of an uncertain LP. We show that using affine decision rules, probabilistic bisimulation relation can be approximated in polynomial time. We have implemented our approach and demonstrate its effectiveness on several case studies.

[1]  L. Ghaoui,et al.  Robust model predictive control through adjustable variables: an application to path planning , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[2]  Elana Guslitser.  UNCERTAINTY- IMMUNIZED SOLUTIONS IN LINEAR PROGRAMMING , 2002 .

[3]  L. Khachiyan.  Polynomial algorithms in linear programming , 1980 .

[4]  Laurent El Ghaoui,et al.  Robust Control of Markov Decision Processes with Uncertain Transition Matrices , 2005, Oper. Res..

[5]  Arkadi Nemirovski,et al.  Robust solutions of uncertain linear programs , 1999, Oper. Res. Lett..

[6]  Marta Z. Kwiatkowska,et al.  Stochastic Transition Systems for Continuous State Spaces and Non-determinism , 2005, FoSSaCS.

[7]  Daniel Gebler,et al.  Computing Behavioral Relations for Probabilistic Concurrent Systems , 2012, ROCKS.

[8]  Moshe Y. Vardi.  Automatic verification of probabilistic concurrent finite state programs , 1985, 26th Annual Symposium on Foundations of Computer Science (sfcs 1985).

[9]  Taolue Chen,et al.  On the complexity of model checking interval-valued discrete time Markov chains , 2013, Inf. Process. Lett..

[10]  Martin Leucker,et al.  Don't Know in Probabilistic Systems , 2006, SPIN.

[11]  Holger Hermanns,et al.  Towards Performance Prediction of Compositional Models in Industrial GALS Designs , 2009, CAV.

[12]  Marc Goerigk,et al.  ROPI—a robust optimization programming interface for C++ , 2014, Optim. Methods Softw..

[13]  Jan Krcál,et al.  Probabilistic Bisimulations for PCTL Model Checking of Interval MDPs (extended version) , 2014, SynCoP.

[14]  L. G. H. Cijan.  A polynomial algorithm in linear programming , 1979 .

[15]  Mahesh Viswanathan,et al.  Model-Checking Markov Chains in the Presence of Uncertainties , 2006, TACAS.

[16]  Johan Löfberg,et al.  Automatic robust convex programming , 2012, Optim. Methods Softw..

[17]  Kim Guldstrand Larsen,et al.  Specification and refinement of probabilistic processes , 1991, [1991] Proceedings Sixth Annual IEEE Symposium on Logic in Computer Science.

[18]  Constantine Caramanis,et al.  Theory and Applications of Robust Optimization , 2010, SIAM Rev..

[19]  Robert Givan,et al.  Bounded Parameter Markov Decision Processes , 1997, ECP.

[20]  Garud Iyengar,et al.  Robust Dynamic Programming , 2005, Math. Oper. Res..

[21]  Holger Hermanns,et al.  Polynomial time decision algorithms for probabilistic automata , 2015, Inf. Comput..

[22]  Kim G. Larsen,et al.  New Results on Abstract Probabilistic Automata , 2011, 2011 Eleventh International Conference on Application of Concurrency to System Design.

[23]  Di Wu,et al.  Reachability analysis of uncertain systems using bounded-parameter Markov decision processes , 2008, Artif. Intell..

[24]  Wang Yi,et al.  Algebraic Reasoning for Real-Time Probabilistic Processes with Uncertain Information , 1994, FTRTFT.

[25]  Ufuk Topcu,et al.  Robust control of uncertain Markov Decision Processes with temporal logic specifications , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[26]  Kim G. Larsen,et al.  Decision Problems for Interval Markov Chains , 2011, LATA.

[27]  Robert E. Tarjan,et al.  Three Partition Refinement Algorithms , 1987, SIAM J. Comput..

[28]  Krishnendu Chatterjee,et al.  Model-Checking omega-Regular Properties of Interval Markov Chains , 2008, FoSSaCS.

[29]  Holger Hermanns,et al.  Deciding probabilistic automata weak bisimulation: theory and practice , 2016, Formal Aspects of Computing.

[30]  Roberto Segala,et al.  Decision Algorithms for Probabilistic Bisimulation , 2002, CONCUR.

[31]  Scott A. Smolka,et al.  CCS expressions, finite state processes, and three problems of equivalence , 1983, PODC '83.

[32]  John N. Tsitsiklis,et al.  Introduction to linear optimization , 1997, Athena scientific optimization and computation series.

[33]  Bengt Jonsson,et al.  A logic for reasoning about time and reliability , 1990, Formal Aspects of Computing.

[34]  K. Subramani,et al.  Compositional Bisimulation Minimization for Interval Markov Decision Processes , 2016, LATA.

[35]  Lijun Zhang,et al.  Synthesis for PCTL in Parametric Markov Decision Processes , 2011, NASA Formal Methods.

[36]  Roberto Segala,et al.  Modeling and verification of randomized distributed real-time systems , 1996 .

[37]  Narendra Karmarkar,et al.  A new polynomial-time algorithm for linear programming , 1984, STOC '84.

[38]  Alberto L. Sangiovanni-Vincentelli,et al.  Polynomial-Time Verification of PCTL Properties of MDPs with Convex Uncertainties , 2013, CAV.

[39]  James Worrell,et al.  LTL Model Checking of Interval Markov Chains , 2013, TACAS.

[40]  Martin Grötschel,et al.  The ellipsoid method and its consequences in combinatorial optimization , 1981, Comb..

[41]  A. Ben-Tal,et al.  Adjustable robust solutions of uncertain linear programs , 2004, Math. Program..

[42]  Laurent El Ghaoui,et al.  Robust Optimization , 2021, ICORES.

[43]  P. Spreij.  Probability and Measure , 1996 .

[44]  George B Dantzig,et al.  ON THE SOLUTION OF TWO-STAGE LINEAR PROGRAMS UNDER UNCERTAINTY. NOTES ON LINEAR PROGRAMMING AND EXTENSIONS. PART 55 , 1961 .

[45]  Alberto Alessandro Angelo Puggelli,et al.  Formal Techniques for the Verification and Optimal Control of Probabilistic Systems in the Presence of Modeling Uncertainties , 2014 .

[46]  Lev V. Utkin,et al.  Interval-Valued Finite Markov Chains , 2002, Reliab. Comput..

[47]  Marta Z. Kwiatkowska,et al.  PRISM 4.0: Verification of Probabilistic Real-Time Systems , 2011, CAV.

[48]  Joost-Pieter Katoen,et al.  Automated compositional Markov chain generation for a plain-old telephone system , 2000, Sci. Comput. Program..