Spoken Instruction-Based One-Shot Object and Action Learning in a Cognitive Robotic Architecture

Learning new knowledge from single instructions and being able to apply it immediately is a highly desirable capability for artificial agents. We provide the first demonstration of spoken instruction-based one-shot object and action learning in a cognitive robotic architecture and discuss the modifications to several architectural components required to enable such fast learning, demonstrating the new capabilities on two different fully autonomous robots.

[1]  Danica Kragic,et al.  Learning a dictionary of prototypical grasp-predicting parts from grasping experience , 2013, 2013 IEEE International Conference on Robotics and Automation.

[2]  Honglak Lee,et al.  Deep learning for detecting robotic grasps , 2013, Int. J. Robotics Res..

[3]  Xiaoping Chen,et al.  Toward open knowledge enabling for human-robot interaction , 2013, HRI 2013.

[4]  Markus Vincze,et al.  Learning grasps for unknown objects in cluttered scenes , 2013, 2013 IEEE International Conference on Robotics and Automation.

[5]  Matthias Scheutz,et al.  First steps toward natural human-like HRI , 2007, Auton. Robots.

[6]  Pietro Perona,et al.  One-shot learning of object categories , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Matthias Scheutz,et al.  Learning to Recognize Novel Objects in One Shot through Human-Robot Interactions in Natural Language Dialogues , 2014, AAAI.

[8]  Matthias Scheutz,et al.  Learning actions from human-robot dialogues , 2011, 2011 RO-MAN.

[9]  Matthias Scheutz,et al.  A parallelized dynamic programming approach to zero resource spoken term discovery , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[10]  Alexander Herzog,et al.  Template-based learning of grasp selection , 2012, 2012 IEEE International Conference on Robotics and Automation.

[11]  Jeannette Bohg,et al.  Leveraging big data for grasp planning , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[12]  Marc Schröder,et al.  Multilingual MARY TTS participation in the Blizzard Challenge 2009 , 2009 .

[13]  John E. Laird,et al.  Flexibly Instructable Agents , 1995, J. Artif. Intell. Res..

[14]  Ashutosh Saxena,et al.  Robotic Grasping of Novel Objects using Vision , 2008, Int. J. Robotics Res..

[15]  Matthias Scheutz.  “Teach one, teach all” - The explosive combination of instructible robots connected via cyber systems , 2014, The 4th Annual IEEE International Conference on Cyber Technology in Automation, Control and Intelligent.

[16]  Michael Beetz,et al.  Everything robots always wanted to know about housework (but were afraid to ask) , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[17]  John E. Laird,et al.  Interactive Task Learning for Simple Games , 2014 .

[18]  Joseph Redmon,et al.  Real-time grasp detection using convolutional neural networks , 2014, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[19]  Çetin Meriçli,et al.  An Interactive Approach for Situated Task Teaching through Verbal Instructions , 2013, AAAI 2013.

[20]  Robert Platt,et al.  Using Geometry to Detect Grasp Poses in 3D Point Clouds , 2015, ISRR.

[21]  Matei T. Ciocarlie,et al.  Mobile Manipulation in Unstructured Environments: Perception, Planning, and Execution , 2012, IEEE Robotics & Automation Magazine.

[22]  John E. Laird,et al.  Learning Grounded Language through Situated Interactive Instruction , 2012, AAAI Fall Symposium: Robots Learning Interactively from Human Teachers.

[23]  Evan A. Krause,et al.  Novel Mechanisms for Natural Human-Robot Interactions in the DIARC Architecture , 2013 .

[24]  Guido Bugmann,et al.  Training Personal Robots Using Natural Language Instruction , 2001, IEEE Intell. Syst..

[25]  Joshua B. Tenenbaum,et al.  Concept learning as motor program induction: A large-scale empirical study , 2012, CogSci.

[26]  Matthias Scheutz,et al.  What to do and how to do it: Translating natural language directives into temporal and dynamic logic representation for goal management and action execution , 2009, 2009 IEEE International Conference on Robotics and Automation.

[27]  Kate Saenko,et al.  High precision grasp pose detection in dense clutter , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[28]  S. Chitta,et al.  Perception , Planning , and Execution for Mobile Manipulation in Unstructured Environments , 2012 .

[29]  Guillaume Aimetti,et al.  Modelling Early Language Acquisition Skills: Towards a General Statistical Learning Mechanism , 2009, EACL.