Semantic-based interaction for teaching robot behavior compositions

Allowing humans to teach robot behaviors will facilitate acceptability as well as long-term interactions. Humans would mainly use speech to transfer knowledge or to teach highlevel behaviors. In this paper, we propose a proof-of-concept application allowing a Pepper robot to learn behaviors from their natural-language-based description, provided by naive human users. In our model, natural language input is provided by grammar-free speech recognition, and is then processed to produce semantic knowledge, grounded in language and primitive behaviors. The same semantic knowledge is used to represent any kind of perceived input as well as actions the robot can perform. The experiment shows that the system can work independently from the domain of application, but also that it has limitations. Progress in semantic extraction, behavior planning and interaction scenario could stretch these limits.

[1]  Moritz Tenorth,et al.  Priming transformational planning with observations of human activities , 2010, 2010 IEEE International Conference on Robotics and Automation.

[2]  Sonia Chernova,et al.  Collaborative Learning of Hierarchical Task Networks from Demonstration and Instruction , 2014, AAAI Fall Symposia.

[3]  Andrea Lockerd Thomaz,et al.  Teachable robots: Understanding human teaching behavior to build more effective robot learners , 2008, Artif. Intell..

[4]  Amit Kumar Pandey,et al.  Human robot interaction can boost robot's affordance learning: A proof of concept , 2015, 2015 International Conference on Advanced Robotics (ICAR).

[5]  Jörg Conradt,et al.  Serendipitous Offline Learning in a Neuromorphic Robot , 2016, Front. Neurorobot..

[6]  Song-Chun Zhu,et al.  Jointly Learning Grounded Task Structures from Language Instruction and Visual Demonstration , 2016, EMNLP.

[7]  Mohamed Chetouani,et al.  Training a robot with evaluative feedback and unlabeled guidance signals , 2016, 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[8]  Manuela M. Veloso,et al.  Using dialog and human observations to dictate tasks to a learning robot assistant , 2008, Intell. Serv. Robotics.

[9]  Charles J. Fillmore,et al.  Frames and the semantics of understanding , 1985 .

[10]  Moritz Tenorth,et al.  Understanding and executing instructions for everyday manipulation tasks from the World Wide Web , 2010, 2010 IEEE International Conference on Robotics and Automation.

[11]  Guido Bugmann,et al.  Personal Robot Training via Natural-Language Instructions. , 2001 .

[12]  Pierre-Yves Oudeyer,et al.  Pragmatic Frames for Teaching and Learning in Human–Robot Interaction: Review and Challenges , 2016, Front. Neurorobot..

[13]  Mihai Surdeanu,et al.  The Stanford CoreNLP Natural Language Processing Toolkit , 2014, ACL.

[14]  Peter Ford Dominey,et al.  Cooperative human robot interaction systems: IV. Communication of shared plans with Naïve humans using gaze and speech , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[15]  Maya Cakmak,et al.  Power to the People: The Role of Humans in Interactive Machine Learning , 2014, AI Mag..

[16]  Jun Tani,et al.  Emergence of Functional Hierarchy in a Multiple Timescale Neural Network Model: A Humanoid Robot Experiment , 2008, PLoS Comput. Biol..

[17]  Guillaume Gibert,et al.  Proof of concept for a user-centered system for sharing cooperative plan knowledge over extended periods and crew changes in space-flight operations , 2015, 2015 24th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[18]  Rachid Alami,et al.  Towards Human-Level Semantics Understanding of Human-Centered Object Manipulation Tasks for HRI: Reasoning About Effect, Ability, Effort and Perspective Taking , 2014, International Journal of Social Robotics.

[19]  Michael Beetz,et al.  Cloud-Based Probabilistic Knowledge Services for Instruction Interpretation , 2015, ISRR.

[20]  Guido Bugmann,et al.  Training Personal Robots Using Natural Language Instruction , 2001, IEEE Intell. Syst..

[21]  Daniele Nardi,et al.  Teaching Robots Parametrized Executable Plans Through Spoken Interaction , 2015, AAMAS.

[22]  Peter Ford Dominey,et al.  Dynamic Construction Grammar and Steps Towards the Narrative Construction of Meaning , 2017, AAAI Spring Symposia.