Modeling Markov Decision Processes with Imprecise Probabilities Using Probabilistic Logic Programming

We study languages that specify Markov Decision Processes with Imprecise Probabilities (MDPIPs) by mixing probabilities and logic programming. We propose a novel language that can capture MDPIPs and Markov Decision Processes with Set-valued Transitions (MDPSTs); we then obtain the complexity of one-step inference for the resulting MDPIPs and MDPSTs. We also present results of independent interest on the complexity of inference with probabilistic logic programs containing interval-valued probabilistic assessments. Finally, we also discuss policy generation techniques.

[1]  Matthias C. M. Troffaes,et al.  Introduction to imprecise probabilities , 2014 .

[2]  Richard Fikes,et al.  STRIPS: A New Approach to the Application of Theorem Proving to Problem Solving , 1971, IJCAI.

[3]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[4]  John Wylie Lloyd,et al.  Foundations of Logic Programming , 1987, Symbolic Computation.

[5]  Denis Deratani Mauá,et al.  The Complexity of Inferences and Explanations in Probabilistic Logic Programming , 2017, ECSQARU.

[6]  F. Cozman,et al.  Representing and solving factored markov decision processes with imprecise probabilities , 2009 .

[7]  Stuart J. Russell,et al.  First-Order Open-Universe POMDPs , 2014, UAI.

[8]  Robert Givan,et al.  Bounded Parameter Markov Decision Processes , 1997, ECP.

[9]  Jacobo Torán,et al.  Complexity classes defined by counting quantifiers , 1991, JACM.

[10]  Craig Boutilier,et al.  Decision-Theoretic, High-Level Agent Programming in the Situation Calculus , 2000, AAAI/IAAI.

[11]  Klaus W. Wagner,et al.  The complexity of combinatorial problems with succinct input representation , 1986, Acta Informatica.

[12]  Fabio Gagliardi Cozman,et al.  Mixed Probabilistic and Nondeterministic Factored Planning through Markov Decision Processes with Set-Valued Transitions , 2008 .

[13]  Luc De Raedt,et al.  Planning in Discrete and Continuous Markov Decision Processes by Probabilistic Programming , 2015, ECML/PKDD.

[14]  Thomas Lukasiewicz,et al.  Probabilistic Reasoning about Actions in Nonmonotonic Causal Theories , 2002, UAI 2002.

[15]  Thomas Lukasiewicz,et al.  Reasoning about actions with sensing under qualitative and probabilistic uncertainty , 2004, TOCL.

[16]  Håkan L. S. Younes,et al.  PPDDL 1 . 0 : An Extension to PDDL for Expressing Planning Domains with Probabilistic Effects , 2004 .

[17]  Denis Deratani Mauá,et al.  Markov Decision Processes Specified by Probabilistic Logic Programming: Representation and Solution , 2016, 2016 5th Brazilian Conference on Intelligent Systems (BRACIS).

[18]  Chelsea C. White,et al.  Markov Decision Processes with Imprecise Transition Probabilities , 1994, Oper. Res..

[19]  Wolfgang Faber,et al.  A logic programming approach to knowledge-state planning: Semantics and complexity , 2004, TOCL.

[20]  Enrico Giunchiglia,et al.  An Action Language Based on Causal Explanation: Preliminary Report , 1998, AAAI/IAAI.

[21]  Osamu Watanabe,et al.  Polynomial Time 1-Turing Reductions from #PH to #P , 1992, Theor. Comput. Sci..

[22]  Denis Deratani Mauá,et al.  The Structure and Complexity of Credal Semantics , 2016, PLP@ILP.

[23]  Thomas Lukasiewicz.  Probabilistic description logic programs , 2007, Int. J. Approx. Reason..

[24]  Hector J. Levesque,et al.  GOLOG: A Logic Programming Language for Dynamic Domains , 1997, J. Log. Program..

[25]  Scott Sanner,et al.  Solutions to Factored MDPs with Imprecise Transition Probabilities 1 , 2011 .

[26]  Joseph Y. Halpern,et al.  Knowledge, probability, and adversaries , 1989, PODC '89.

[27]  J. K. Satia,et al.  Markovian Decision Processes with Uncertain Transition Probabilities , 1973, Oper. Res..

[28]  Fabio Gagliardi Cozman,et al.  The Inferential Complexity of Bayesian and Credal Networks , 2005, IJCAI.

[29]  Enrico Giunchiglia,et al.  Nonmonotonic causal theories , 2004, Artif. Intell..

[30]  Fabio Gagliardi Cozman,et al.  Planning under Risk and Knightian Uncertainty , 2007, IJCAI.

[31]  Fabio Gagliardi Cozman,et al.  Graphical models for imprecise probabilities , 2005, Int. J. Approx. Reason..

[32]  Luc De Raedt,et al.  Logical Markov Decision Programs , 2003 .