Closed-Loop Robot Task Planning Based on Referring Expressions

Increasing the accessibility of autonomous robots also for inexperienced users requires user-friendly and high-level control opportunities of robotic systems. While automated planning is able to decompose a complex task into a sequence of steps which reaches an intended goal, it is difficult to formulate such a goal without knowing the internals of the planning system and the exact capabilities of the robot. This becomes even more important in dynamic environments in which manipulable objects are subject to change. In this paper, we present an adaptive control interface which allows users to specify goals based on an internal world model by incrementally building referring expressions to the objects in the world. We consider fetch-and-carry tasks and automatically deduce potential high-level goals from the world model to make them available to the user. Based on its perceptions our system can react to changes in the environment by adapting the goal formulation within the domain-independent planning system.

[1]  Ehud Reiter,et al.  Book Reviews: Building Natural Language Generation Systems , 2000, CL.

[2]  Wolfram Burgard,et al.  Learning mobile manipulation actions from human demonstrations , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[3]  Leslie Pack Kaelbling,et al.  A constraint-based method for solving sequential manipulation planning problems , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[4]  Danica Kragic,et al.  SimTrack: A simulation-based framework for scalable real-time object pose detection and tracking , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[5]  Rachid Alami,et al.  Towards Combining HTN Planning and Geometric Task Planning , 2013, ArXiv.

[6]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[7]  John E. Laird,et al.  Grounding Language for Interactive Task Learning , 2017, RoboNLP@ACL.

[8]  Wolfram Burgard,et al.  BI2RRT*: An efficient sampling-based path planning framework for task-constrained mobile manipulation , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[9]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[10]  Pieter Abbeel,et al.  Combined task and motion planning through an extensible planner-independent interface layer , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[11]  Swarat Chaudhuri,et al.  Incremental Task and Motion Planning: A Constraint-Based Approach , 2016, Robotics: Science and Systems.

[12]  Leslie Pack Kaelbling,et al.  Hierarchical task and motion planning in the now , 2011, 2011 IEEE International Conference on Robotics and Automation.

[13]  Robert Mattmüller,et al.  Using the Context-enhanced Additive Heuristic for Temporal and Numeric Planning , 2009, ICAPS.

[14]  Leslie Pack Kaelbling,et al.  Integrated task and motion planning in belief space , 2013, Int. J. Robotics Res..

[15]  Blai Bonet,et al.  A Robust and Fast Action Selection Mechanism for Planning , 1997, AAAI/IAAI.

[16]  Malte Helmert,et al.  The Fast Downward Planning System , 2006, J. Artif. Intell. Res..

[17]  Wolfram Burgard,et al.  Acting thoughts: Towards a mobile robotic service assistant for users with limited communication skills , 2017, 2017 European Conference on Mobile Robots (ECMR).

[18]  Bernhard Nebel,et al.  Semantic Attachments for Domain-Independent Planning Systems , 2009, ICAPS.

[19]  Licheng Yu,et al.  Modeling Context in Referring Expressions , 2016, ECCV.

[20]  Ronald P. A. Petrick,et al.  EXPERIENCES WITH PLANNING FOR NATURAL LANGUAGE GENERATION , 2011, Comput. Intell..

[21]  Mohit Shridhar,et al.  Grounding Spatio-Semantic Referring Expressions for Human-Robot Interaction , 2017, ArXiv.

[22]  Moritz Göbelbecker.  Assisting wit Goal Formulation for Domain Independent Planning , 2015, KI.

[23]  Robert Dale,et al.  Computational Interpretations of the Gricean Maxims in the Generation of Referring Expressions , 1995, Cogn. Sci..