Addressing Uncertainty in Hierarchical User-Centered Planning

Companion-Systems need to reason about dynamic properties of their users, e.g., their emotional state, and the current state of the environment. The values of these properties are often not directly accessible; hence information on them must be pieced together from indirect, noisy or partial observations. To ensure probability-based treatment of partial observability on the planning level, planning problems can be modeled as Partially Observable Markov Decision Processes (POMDPs).

[1]  Susanne Biundo-Stephan,et al.  Exploiting Expert Knowledge in Factored POMDPs , 2012, ECAI.

[2]  James A. Hendler,et al.  UMCP: A sound and complete planning procedure for hierarchical task-network planning , 1994 .

[3]  Joelle Pineau,et al.  Policy-contingent abstraction for robust robot control , 2002, UAI.

[4]  Thomas G. Dietterich.  Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition , 1999, J. Artif. Intell. Res..

[5]  Pascal Bercher,et al.  On the Decidability of HTN Planning with Task Insertion , 2011, IJCAI.

[6]  Doina Precup,et al.  Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning , 1999, Artif. Intell..

[7]  Malte Helmert,et al.  Trial-Based Heuristic Tree Search for Finite Horizon MDPs , 2013, ICAPS.

[8]  E. J. Sondik,et al.  The Optimal Control of Partially Observable Markov Decision Processes. , 1971 .

[9]  Florian Nothdurft,et al.  Plan, Repair, Execute, Explain - How Planning Helps to Assemble your Home Theater , 2014, ICAPS.

[10]  Simon M. Lucas,et al.  A Survey of Monte Carlo Tree Search Methods , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[11]  Nicholas Roy,et al.  PUMA: Planning Under Uncertainty with Macro-Actions , 2010, AAAI.

[12]  Gregor Bertrand,et al.  Companion-Technology: Towards User- and Situation-Adaptive Functionality of Technical Systems , 2014, 2014 International Conference on Intelligent Environments.

[13]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[14]  Eric A. Hansen,et al.  Synthesis of Hierarchical Finite-State Controllers for POMDPs , 2003, ICAPS.

[15]  Susanne Biundo-Stephan,et al.  HTN-Style Planning in Relational POMDPs Using First-Order FSCs , 2011, KI.

[16]  Wolfgang Minker,et al.  A Planning-Based Assistance System for Setting Up a Home Theater , 2015, AAAI.

[17]  Joel Veness,et al.  Monte-Carlo Planning in Large POMDPs , 2010, NIPS.

[18]  Leslie Pack Kaelbling,et al.  Approximate Planning in POMDPs with Macro-Actions , 2003, NIPS.

[19]  Hector Muñoz-Avila,et al.  Applications of SHOP and SHOP2 , 2005, IEEE Intelligent Systems.

[20]  Stuart J. Russell,et al.  Reinforcement Learning with Hierarchies of Machines , 1997, NIPS.