Generating Task-Oriented Interactions of Service Robots

Human-robot interaction (HRI) may play a key role in enhancing a robot's capability in practical service tasks, allowing productive human-robot collaboration. To obtain appropriate human aid for conducting tasks, a robot should be capable of generating meaningful questions regarding the task procedures in real time and applying the results to modify its task plans or behaviors. However, few studies on integrating robot task management and HRI in such high-level task planning exist. In this paper, we propose a new scheme of script-based task planning and HRI that supports the planning and is generated by it. The planning operates on a set of plain and easily writable task procedures, or scripts. The approach produces robust, practical, and easy-to-manipulate robot behavior. Based on the scripts, the system identifies plan ambiguities that require interaction with humans, and resolves them using the human response. The interaction thus generated is highly relevant and task-oriented. The robot learns from the interaction history to improve its subsequent planning in general, or personalize it. Two simulation cases, of a home service and a museum-guide robot, are presented to show how the robots lead appropriate interaction and smoothly adjust their task plans to user commands or responses. Such close integration of HRI and robot task planning is expected to advance the practicality of service robots.

[1]  Joelle Pineau,et al.  Development and Validation of a Robust Speech Interface for Improved Human-Robot Interaction , 2009, Int. J. Soc. Robotics.

[2]  Odest Chadwicke Jenkins,et al.  Modeling Aspects of Theory of Mind with Markov Random Fields , 2009, Int. J. Soc. Robotics.

[3]  Christopher G. Atkeson,et al.  Predicting human interruptibility with sensors , 2005, TCHI.

[4]  Erik L L Sonnhammer,et al.  Quality assessment of multiple alignment programs , 2002, FEBS letters.

[5]  Alexei Makarenko,et al.  Human-robot communication for collaborative decision making - A probabilistic approach , 2010, Robotics Auton. Syst..

[6]  Yo Chan Kim,et al.  Designing a Support System for Action Rule Extraction in Script-Based Robot Action Planning , 2008 .

[7]  Luca Spalazzi,et al.  A Survey on Case-Based Planning , 2004, Artificial Intelligence Review.

[8]  Alexandra Kirsch,et al.  Learning Ability Models for Human-Robot Collaboration , 2010, RSS 2010.

[9]  Stephanie Rosenthal,et al.  Is Someone in this Office Available to Help Me? , 2012, J. Intell. Robotic Syst..

[10]  Sung-Bae Cho,et al.  Mixed-Initiative Human–Robot Interaction Using Hierarchical Bayesian Networks , 2007, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[11]  Michael A. Goodrich,et al.  Human-Robot Interaction: A Survey , 2008, Found. Trends Hum. Comput. Interact..

[12]  Jeff Orkin,et al.  The Restaurant Game: Learning Social Behavior and Language from Thousands of Players Online , 2008, J. Game Dev..

[13]  Cynthia Breazeal,et al.  Cost-Based Anticipatory Action Selection for Human–Robot Fluency , 2007, IEEE Transactions on Robotics.

[14]  Ronald C. Arkin,et al.  Usability evaluation of high-level user assistance for robot mission specification , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[15]  Cynthia Breazeal,et al.  Crowdsourcing human-robot interaction: Application from virtual to physical worlds , 2011, 2011 RO-MAN.

[16]  Il Hong Suh,et al.  Ontology-Based Unified Robot Knowledge for Service Robots in Indoor Environments , 2011, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[17]  Rüdiger Dillmann,et al.  Learning sequential constraints of tasks from user demonstrations , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[18]  Ehud Reiter,et al.  Book Reviews: Building Natural Language Generation Systems , 2000, CL.

[19]  Christopher J. Lee,et al.  Combining partial order alignment and progressive multiple sequence alignment increases alignment speed and scalability to very large alignment problems , 2004, Bioinform..

[20]  Martin Buss,et al.  Human-Robot Collaboration: a Survey , 2008, Int. J. Humanoid Robotics.

[21]  Matthias Scheutz,et al.  Planning for Human-Robot Teaming , 2011 .

[22]  Alois Knoll,et al.  Human-Robot dialogue for joint construction tasks , 2006, ICMI '06.

[23]  Yuichiro Anzai,et al.  Task planning based on human-robot interaction for autonomous mobile robots , 1996, Proceedings 5th IEEE International Workshop on Robot and Human Communication. RO-MAN'96 TSUKUBA.

[24]  Matthias Scheutz,et al.  Tell me when and why to do it! Run-time planner model updates via natural language instruction , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[25]  Anders Orebäck,et al.  BERRA: a research architecture for service robots , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[26]  Guido Bugmann,et al.  Mobile robot programming using natural language , 2002, Robotics Auton. Syst..

[27]  M S Waterman,et al.  Identification of common molecular subsequences. , 1981, Journal of molecular biology.

[28]  Rüdiger Dillmann,et al.  A Flexible Task Knowledge Representation for Service Robots , 2006, IAS.

[29]  C. Galindo,et al.  Control Architecture for Human–Robot Integration: Application to a Robotic Wheelchair , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[30]  Robin R. Murphy,et al.  Human-robot interaction in rescue robotics , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[31]  Christian Wallraven,et al.  The POETICON Corpus: Capturing Language Use and Sensorimotor Experience in Everyday Interaction , 2010, LREC.

[32]  Andrew T. Irish,et al.  Trajectory Learning for Robot Programming by Demonstration Using Hidden Markov Model and Dynamic Time Warping , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[33]  Stephanie Rosenthal,et al.  An effective personal mobile robot agent through symbiotic human-robot interaction , 2010, AAMAS.

[34]  Rachid Alami,et al.  A Task Planner for an Autonomous Social Robot , 2008, DARS.

[35]  Bruce A. MacDonald,et al.  Generalizing topological task graphs from multiple symbolic demonstrations in programming by demonstration (PbD) processes , 2011, 2011 IEEE International Conference on Robotics and Automation.

[36]  Danica Kragic,et al.  Robot Learning from Demonstration: A Task-level Planning Approach , 2008 .

[37]  Rachid Alami,et al.  Plan-Based Control of Joint Human-Robot Activities , 2010, KI - Künstliche Intelligenz.

[38]  Naoyuki Kanda,et al.  A two-layer model for behavior and dialogue planning in conversational service robots , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[39]  James Fogarty,et al.  Presence versus availability: the design and evaluation of a context-aware communication client , 2004, Int. J. Hum. Comput. Stud..

[40]  Rüdiger Dillmann,et al.  From Abstract Task Knowledge to Executable Robot Programs , 2008, J. Intell. Robotic Syst..

[41]  Nikolaos Mavridis,et al.  To ask or to sense? Planning to integrate speech and sensorimotor acts , 2012, 2012 IV International Congress on Ultra Modern Telecommunications and Control Systems.

[42]  Monica N. Nicolescu,et al.  Learning and interacting in human-robot domains , 2001, IEEE Trans. Syst. Man Cybern. Part A.

[43]  Dong-Seok Lee,et al.  Coupling structural and functional models for interaction design , 2004, Interact. Comput..

[44]  Woojin Chung,et al.  Tripodal Schematic Control Architecture for Integration of Multi-Functional Indoor Service Robots , 2006, IEEE Transactions on Industrial Electronics.

[45]  Charles E. Thorpe,et al.  Collaborative control: a robot-centric model for vehicle teleoperation , 2001 .

[46]  Richard Fikes,et al.  STRIPS: A New Approach to the Application of Theorem Proving to Problem Solving , 1971, IJCAI.

[47]  Kim J. Vicente,et al.  Ecological interface design: theoretical foundations , 1992, IEEE Trans. Syst. Man Cybern..

[48]  Robert Dale,et al.  Building Natural Language Generation Systems: Figures , 2000 .

[49]  Matthias Scheutz,et al.  Planning for Agents with Changing Goals , 2011 .

[50]  Laurel D. Riek,et al.  A decomposition of UAV-related situation awareness , 2006, HRI '06.

[51]  James A. Hendler,et al.  AI Planning: Systems and Techniques , 1990, AI Mag..

[52]  Reid G. Simmons,et al.  A task description language for robot control , 1998, Proceedings. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems. Innovations in Theory, Practice and Applications (Cat. No.98CH36190).

[53]  Cory D. Kidd,et al.  HUMANOID ROBOTS AS COOPERATIVE PARTNERS FOR PEOPLE , 2004 .

[54]  Terrence Fong,et al.  Robot, asker of questions , 2003, Robotics Auton. Syst..

[55]  Luca Spalzzi,et al.  A Survey on Case-Based Planning , 2001 .

[56]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[57]  Deb Roy,et al.  Grounded Situation Models for Robots: Where words and percepts meet , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[58]  Stephanie Rosenthal,et al.  Mixed-Initiative Long-Term Interactions with an All-Day-Companion Robot , 2010, AAAI Fall Symposium: Dialog with Robots.

[59]  Christopher J. Lee,et al.  Multiple sequence alignment using partial order graphs , 2002, Bioinform..

[60]  Cipriano Galindo,et al.  Multihierarchical Interactive Task Planning: Application to Mobile Robotics , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[61]  Yochan Kim,et al.  Managing scripts for interactive service robots by using action-relation rules , 2010, IFAC HMS.

[62]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[63]  Rosalind W. Picard,et al.  Grounded situation models for situated conversational assistants , 2007 .

[64]  Brian Scassellati,et al.  Theory of Mind for a Humanoid Robot , 2002, Auton. Robots.