Magnifying-Lens Abstraction for Markov Decision Processes

We present a novel abstraction technique which allows the analysis of reachability and safety properties of Markov decision processes with very large state spaces. The technique, called magnifying-lens abstraction, (MLA) copes with the state-explosion problem by partitioning the state-space into regions, and by computing upper and lower bounds for reachability and safety properties on the regions, rather than on the states. To compute these bounds, MLA iterates over the regions, considering the concrete states of each region in turn, as if one were sliding across the abstraction a magnifying lens which allowed viewing the concrete states. The algorithm adaptively refines the regions, using smaller regions where more detail is needed, until the difference between upper and lower bounds is smaller than a specified accuracy. We provide experimental results on three case studies illustrating that MLA can provide accurate answers, with savings in memory requirements.

[1]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[2]  Martin Leucker,et al.  Don't Know in Probabilistic Systems , 2006, SPIN.

[3]  Itzhak Gilboa,et al.  Additive representations of non-additive measures and the choquet integral , 1994, Ann. Oper. Res..

[4]  Moshe Y. Vardi.  Automatic verification of probabilistic concurrent finite state programs , 1985, 26th Annual Symposium on Foundations of Computer Science (sfcs 1985).

[5]  Luca de Alfaro,et al.  Computing Minimum and Maximum Reachability Times in Probabilistic Systems , 1999, CONCUR.

[6]  Annabelle McIver,et al.  Abstraction, Refinement And Proof For Probabilistic Systems (Monographs in Computer Science) , 2004 .

[7]  Robert Givan,et al.  Model Minimization in Markov Decision Processes , 1997, AAAI/IAAI.

[8]  Michael Huth,et al.  On finite-state approximants for probabilistic computation tree logic , 2005, Theor. Comput. Sci..

[9]  Masahiro Fujita,et al.  Multi-Terminal Binary Decision Diagrams: An Efficient Data Structure for Matrix Representation , 1997, Formal Methods Syst. Des..

[10]  Stuart Cheshire,et al.  Dynamic Configuration of IPv4 Link-Local Addresses , 2005, RFC.

[11]  Cyrus Derman,et al.  Finite State Markovian Decision Processes , 1970 .

[12]  D. Schmeidler.  Integral representation without additivity , 1986 .

[13]  R. I. Bahar,et al.  Algebraic decision diagrams and their applications , 1993, Proceedings of 1993 International Conference on Computer Aided Design (ICCAD).

[14]  Andreas Podelski,et al.  ACSAR: Software Model Checking with Transfinite Refinement , 2007, SPIN.

[15]  Marta Z. Kwiatkowska,et al.  Game-based Abstraction for Markov Decision Processes , 2006, Third International Conference on the Quantitative Evaluation of Systems - (QEST'06).

[16]  L. D. Alfaro.  The Verification of Probabilistic Systems Under Memoryless Partial-Information Policies is Hard , 1999 .

[17]  Annabelle McIver,et al.  Abstraction, Refinement and Proof for Probabilistic Systems , 2004, Monographs in Computer Science.

[18]  Antti Valmari,et al.  \emph{Don't know} in probabilistic systems , 2005 .

[19]  Luca de Alfaro,et al.  Symbolic Model Checking of Probabilistic Processes Using MTBDDs and the Kronecker Representation , 2000, TACAS.

[20]  I. Gilboa.  Expected utility with purely subjective non-additive probabilities , 1987 .

[21]  David Monniaux.  Abstract interpretation of programs as Markov decision processes , 2005, Sci. Comput. Program..

[22]  Rajeev Alur,et al.  A Temporal Logic of Nested Calls and Returns , 2004, TACAS.

[23]  Bengt Jonsson,et al.  CONCUR ’94: Concurrency Theory , 1994, Lecture Notes in Computer Science.

[24]  M. Berger,et al.  Adaptive mesh refinement for hyperbolic partial differential equations , 1982 .

[25]  Zohar Manna,et al.  The Temporal Logic of Reactive and Concurrent Systems , 1991, Springer New York.

[26]  Roberto Segala,et al.  Modeling and verification of randomized distributed real-time systems , 1996 .

[27]  Insup Lee,et al.  Weak Bisimulation for Probabilistic Systems , 2000, CONCUR.

[28]  Henrik Ejersbo Jensen,et al.  Reachability Analysis of Probabilistic Systems by Successive Refinements , 2001, PAPM-PROBMIV.

[29]  C. A. Petri,et al.  Concurrency Theory , 1986, Advances in Petri Nets.

[30]  Enrico Macii,et al.  Algebric Decision Diagrams and Their Applications , 1997, ICCAD '93.

[31]  Rupak Majumdar,et al.  Quantitative solution of omega-regular games , 2004, J. Comput. Syst. Sci..

[32]  Brigitte Plateau.  On the stochastic structure of parallelism and synchronization models for distributed algorithms , 1985, SIGMETRICS 1985.

[33]  Andrew Hinton,et al.  PRISM: A Tool for Automatic Verification of Probabilistic Systems , 2006, TACAS.

[34]  Nancy A. Lynch,et al.  Probabilistic Simulations for Probabilistic Processes , 1994, Nord. J. Comput..

[35]  Christel Baier,et al.  Weak Bisimulation for Fully Probabilistic Processes , 1997, FBT.

[36]  Kousha Etessami,et al.  Optimizing Büchi Automata , 2000, CONCUR.