Active Inference and Behavior Trees for Reactive Action Planning and Execution in Robotics

This paper presents how the hybrid combination of behavior trees and the neuroscientific principle of active inference can be used for action planning and execution for reactive robot behaviors in dynamic environments. We show how complex robotic tasks can be formulated as a free-energy minimisation problem, and how state estimation and symbolic decision making are handled within the same framework. The general behavior is specified offline through behavior trees, where the leaf nodes represent desired states, not actions as in classical behavior trees. The decision of which action to execute to reach a state is left to the online active inference routine, in order to resolve unexpected contingencies. This hybrid combination improves the robustness of plans specified through behavior trees, while allowing to cope with the curse of dimensionality in active inference. The properties of the proposed algorithm are analysed in terms of robustness and convergence, and the theoretical results are validated using a mobile manipulator in a retail environment.

[1]  James A. Hendler,et al.  HTN Planning: Complexity and Expressivity , 1994, AAAI.

[2]  Daniel E. Koditschek,et al.  Sequential Composition of Dynamically Dexterous Robot Behaviors , 1999, Int. J. Robotics Res..

[3]  Jeff Orkin,et al.  Applying Goal-Oriented Action Planning to Games , 2008 .

[4]  Karl J. Friston The free-energy principle: a unified brain theory? , 2010, Nature Reviews Neuroscience.

[5]  Karl J. Friston,et al.  Action and behavior: a free-energy formulation , 2010, Biological Cybernetics.

[6]  Leslie Pack Kaelbling,et al.  Hierarchical task and motion planning in the now , 2011, 2011 IEEE International Conference on Robotics and Automation.

[7]  Karl J. Friston,et al.  Action understanding and active inference , 2011, Biological Cybernetics.

[8]  Karl J. Friston,et al.  Active inference and agency: optimal control without cost functions , 2012, Biological Cybernetics.

[9]  Leslie Pack Kaelbling,et al.  Integrated task and motion planning in belief space , 2013, Int. J. Robotics Res..

[10]  Leslie Pack Kaelbling,et al.  Foresight and reconsideration in hierarchical planning and execution , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[11]  Marco Aiello,et al.  An Overview of Hierarchical Task Network Planning , 2014, ArXiv.

[12]  Paolo Traverso,et al.  The actor's view of automated planning and acting: A position paper , 2014, Artif. Intell..

[13]  Paolo Traverso,et al.  Blended Planning and Acting: Preliminary Approach, Research Challenges , 2015, AAAI.

[14]  Karl J. Friston,et al.  Active inference and epistemic value , 2015, Cognitive neuroscience.

[15]  Robert Babuska,et al.  An application of sequential composition control to cooperative systems , 2015, 2015 10th International Workshop on Robot Motion and Control (RoMoCo).

[16]  Natàlia Hurtós,et al.  ROSPlan: Planning in the Robot Operating System , 2015, ICAPS.

[17]  Karl J. Friston,et al.  Neuroscience and Biobehavioral Reviews , 2022 .

[18]  Paolo Traverso,et al.  Automated Planning and Acting , 2016 .

[19]  Karl J. Friston,et al.  Active Inference: A Process Theory , 2017, Neural Computation.

[20]  Karl J. Friston,et al.  The graphical brain: Belief propagation and active inference , 2017, Network Neuroscience.

[21]  Petter Ögren,et al.  How Behavior Trees Modularize Hybrid Control Systems and Generalize Sequential Behavior Compositions, the Subsumption Architecture, and Decision Trees , 2017, IEEE Transactions on Robotics.

[22]  Simon McGregor,et al.  The free energy principle for action and perception: A mathematical review , 2017, 1705.09156.

[23]  Petter Ögren,et al.  Behavior Trees in Robotics and AI: An Introduction , 2017, ArXiv.

[24]  Rafal Bogacz,et al.  A tutorial on the free-energy framework for modelling perception and learning , 2017, Journal of mathematical psychology.

[25]  Karl J. Friston,et al.  Computational mechanisms of curiosity and goal-directed exploration , 2018, bioRxiv.

[26]  Karl J. Friston,et al.  Planning and navigation as active inference , 2017, Biological Cybernetics.

[27]  Karl J. Friston,et al.  The Markov blankets of life: autonomy, active inference and the free energy principle , 2018, Journal of The Royal Society Interface.

[28]  Stefan J. Kiebel,et al.  Active Inference, Belief Propagation, and the Bethe Approximation , 2018, Neural Computation.

[29]  Gordon Cheng,et al.  Active inference body perception and action for humanoid robots , 2019, ArXiv.

[30]  Beren Millidge Combining Active Inference and Hierarchical Predictive Coding: A Tutorial Introduction and Case Study , 2019 .

[31]  Dieter Fox,et al.  Representing Robot Task Plans as Robust Logical-Dynamical Systems , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[32]  Karl J. Friston A free energy principle for a particular physics , 2019, 1906.10184.

[33]  Petter Ögren,et al.  Towards Blended Reactive Planning and Acting using Behavior Trees , 2016, 2019 International Conference on Robotics and Automation (ICRA).

[34]  Michele Colledanchise,et al.  Task Planning with Belief Behavior Trees , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[35]  Riccardo M. G. Ferrari,et al.  A Novel Adaptive Controller for Robot Manipulators Based on Active Inference , 2019, IEEE Robotics and Automation Letters.

[36]  Online Replanning in Belief Space for Partially Observable Task and Motion Problems , 2019, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[37]  Martijn Wisse,et al.  Free Energy Principle Based State and Input Observer Design for Linear Systems with Colored Noise , 2020, 2020 American Control Conference (ACC).

[38]  Karl J. Friston,et al.  Active inference on discrete state-spaces: A synthesis , 2020, Journal of mathematical psychology.

[39]  Riccardo M. G. Ferrari,et al.  Active Inference for Fault Tolerant Control of Robot Manipulators with Sensory Faults , 2020 .

[40]  Vicente Matellán Olivera,et al.  Optimized Execution of PDDL Plans using Behavior Trees , 2021, ArXiv.

[41]  Gordon Cheng,et al.  An Empirical Study of Active Inference on a Humanoid Robot , 2021, IEEE Transactions on Cognitive and Developmental Systems.

[42]  Nick Hawes,et al.  Active Inference for Integrated State-Estimation, Control, and Learning , 2020, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[43]  Karl J. Friston,et al.  Active Inference: Demystified and Compared , 2019, Neural Computation.

[44]  Nick Hawes,et al.  Fault-tolerant Control of Robot Manipulators with Sensory Faults using Unbiased Active Inference , 2021, 2021 European Control Conference (ECC).