Object Learning Through Active Exploration

This paper addresses the problem of active object learning by a humanoid child-like robot, using a developmental approach. We propose a cognitive architecture where the visual representation of the objects is built incrementally through active exploration. We present the design guidelines of the cognitive architecture, its main functionalities, and we outline the cognitive process of the robot by showing how it learns to recognize objects in a human-robot interaction scenario inspired by social parenting. The robot actively explores the objects through manipulation, driven by a combination of social guidance and intrinsic motivation. Besides the robotics and engineering achievements, our experiments replicate some observations about the coupling of vision and manipulation in infants, particularly how they focus on the most informative objects. We discuss the further benefits of our architecture, particularly how it can be improved and used to ground concepts.

[1]  Pierre-Yves Oudeyer,et al.  Intrinsic Motivation Systems for Autonomous Mental Development , 2007, IEEE Transactions on Evolutionary Computation.

[2]  Rachid Alami,et al.  Towards a platform-independent cooperative human-robot interaction system: II. Perception, execution and imitation of goal directed actions , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Giulio Sandini,et al.  A Survey of Artificial Cognitive Systems: Implications for the Autonomous Development of Mental Capabilities in Computational Agents , 2007, IEEE Transactions on Evolutionary Computation.

[4]  C. Kemp,et al.  What Can I Control ? : The Development of Visual Categories for a Robot ’ s Body and the World that it Influences , 2006 .

[5]  Luc De Raedt,et al.  Learning relational affordance models for robots in multi-object manipulation tasks , 2012, 2012 IEEE International Conference on Robotics and Automation.

[6]  Manuel Lopes,et al.  Learning Object Affordances: From Sensory--Motor Coordination to Imitation , 2008, IEEE Transactions on Robotics.

[7]  Mark Steedman,et al.  Learning STRIPS Operators from Noisy and Incomplete Observations , 2012, UAI.

[8]  Geoffrey E. Hinton,et al.  Two Distributed-State Models For Generating High-Dimensional Time Series , 2011, J. Mach. Learn. Res..

[9]  Kenneth O. Stanley,et al.  Abandoning Objectives: Evolution Through the Search for Novelty Alone , 2011, Evolutionary Computation.

[10]  Andrea L. Thomaz,et al.  Socially guided machine learning , 2006 .

[11]  Candace L. Sidner,et al.  Recognizing engagement in human-robot interaction , 2010, 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[12]  Alexandre Bernardino,et al.  Multimodal saliency-based bottom-up attention a framework for the humanoid robot iCub , 2008, 2008 IEEE International Conference on Robotics and Automation.

[13]  Armando J. Pinho,et al.  An Ontology-based Multi-level Robot Architecture for Learning from Experiences , 2013, AAAI Spring Symposium: Designing Intelligent Robots.

[14]  J. Konczak,et al.  The development toward stereotypic arm kinematics during reaching in the first 3 years of life , 1997, Experimental Brain Research.

[15]  Pierre-Yves Oudeyer,et al.  Active choice of teachers, learning strategies and goals for a socially guided intrinsic motivation learner , 2012, Paladyn J. Behav. Robotics.

[16]  Jeffrey J. Lockman,et al.  Infant Manual Exploration of Objects, Surfaces, and Their Interrelations , 2005 .

[17]  H. Ruff,et al.  The differentiation of activity in infants' exploration of objects , 1992 .

[18]  M. Tomasello,et al.  Modeling children's early grammatical knowledge , 2009, Proceedings of the National Academy of Sciences.

[19]  Andrea Lockerd Thomaz,et al.  Learning from human teachers with Socially Guided Exploration , 2008, 2008 IEEE International Conference on Robotics and Automation.

[20]  Ruslan Salakhutdinov,et al.  Learning Deep Generative Models , 2009 .

[21]  Charles Kemp,et al.  How to Grow a Mind: Statistics, Structure, and Abstraction , 2011, Science.

[22]  Giulio Sandini,et al.  Learning about objects through action - initial steps towards artificial cognition , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[23]  Pierre-Yves Oudeyer,et al.  The strategic student approach for life-long exploration and learning , 2012, 2012 IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL).

[24]  Mark Steedman,et al.  Object-Action Complexes: Grounded abstractions of sensory-motor processes , 2011, Robotics Auton. Syst..

[25]  Michael Gasser,et al.  The Development of Embodied Cognition: Six Lessons from Babies , 2005, Artificial Life.

[26]  Pierre-Yves Oudeyer,et al.  Learning to recognize objects through curiosity-driven manipulation with the iCub humanoid robot , 2013, 2013 IEEE Third Joint International Conference on Development and Learning and Epigenetic Robotics (ICDL).

[27]  Dirk Kraft,et al.  A Survey of the Ontogeny of Tool Use: From Sensorimotor Experience to Planning , 2013, IEEE Transactions on Autonomous Mental Development.

[28]  Olaf Sporns,et al.  Mapping Information Flow in Sensorimotor Networks , 2006, PLoS Comput. Biol..

[29]  E. Deci,et al.  Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions. , 2000, Contemporary educational psychology.

[30]  Giulio Sandini,et al.  Computing robot internal/external wrenches by means of inertial, tactile and F/T sensors: Theory and implementation on the iCub , 2011, 2011 11th IEEE-RAS International Conference on Humanoid Robots.

[31]  Nitish Srivastava,et al.  Multimodal learning with deep Boltzmann machines , 2012, J. Mach. Learn. Res..

[32]  Peter Ford Dominey,et al.  Multi-modal convergence maps: from body schema and self-representation to mental imagery , 2013, Adapt. Behav..

[33]  Alexander Dekhtyar,et al.  Information Retrieval , 2018, Lecture Notes in Computer Science.

[34]  María Malfaz,et al.  A Biologically Inspired Architecture for an Autonomous and Social Robot , 2011, IEEE Transactions on Autonomous Mental Development.

[35]  Lisa M. Oakes,et al.  Manual object exploration and learning about object features in human infants , 2012, 2012 IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL).

[36]  Maya Cakmak,et al.  Learning about objects with human teachers , 2009, 2009 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[37]  Olivier Sigaud,et al.  Learning compact parameterized skills with a single regression , 2013, 2013 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids).

[38]  Mohamed Chetouani,et al.  Multimodal People Engagement with iCub , 2012, BICA.

[39]  Kelly L. Madole,et al.  The relation between infants' activity with objects and attention to object appearance. , 2008, Developmental psychology.

[40]  Danica Kragic,et al.  Birth of the Object: Detection of Objectness and Extraction of Object Shape through Object-Action complexes , 2008, Int. J. Humanoid Robotics.

[41]  Marco Mirolli,et al.  Intrinsically Motivated Learning in Natural and Artificial Systems , 2013 .

[42]  Richard Alan Peters,et al.  The sensory ego-sphere: a mediating interface between sensors and cognition , 2009, Auton. Robots.

[43]  Derek C. Rose,et al.  Deep Machine Learning - A New Frontier in Artificial Intelligence Research [Research Frontier] , 2010, IEEE Computational Intelligence Magazine.

[44]  Charles Tijus,et al.  Perceiving Action from Static Images: The Role of Spatial Context , 2003, CONTEXT.

[45]  Yukie Nagai,et al.  Learning to grasp with parental scaffolding , 2011, 2011 11th IEEE-RAS International Conference on Humanoid Robots.

[46]  Jana Kosecka,et al.  Semantic segmentation of street scenes by superpixel co-occurrence and 3D geometry , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[47]  Pierre-Yves Oudeyer,et al.  Bootstrapping intrinsically motivated learning with human demonstration , 2011, 2011 IEEE International Conference on Development and Learning (ICDL).

[48]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[49]  Joshua B. Tenenbaum,et al.  Learning to Learn with Compound HD Models , 2011, NIPS.

[50]  Juhan Nam,et al.  Multimodal Deep Learning , 2011, ICML.

[51]  E. Bushnell,et al.  Motor development and the mind: the potential role of motor abilities as a determinant of aspects of perceptual development. , 1993, Child development.

[52]  Emre Ugur,et al.  Goal emulation and planning in perceptual space using learned affordances , 2011, Robotics Auton. Syst..

[53]  Jun Morimoto,et al.  On-line motion synthesis and adaptation using a trajectory database , 2012, Robotics Auton. Syst..

[54]  Barbara Caputo,et al.  Using Object Affordances to Improve Object Recognition , 2011, IEEE Transactions on Autonomous Mental Development.

[55]  Giulio Sandini,et al.  The iCub Platform: A Tool for Studying Intrinsically Motivated Learning , 2013, Intrinsically Motivated Learning in Natural and Artificial Systems.

[56]  Moritz Tenorth,et al.  Representation and Exchange of Knowledge About Actions, Objects, and Environments in the RoboEarth Framework , 2013, IEEE Transactions on Automation Science and Engineering.

[57]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[58]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[59]  Masaki Ogino,et al.  Cognitive Developmental Robotics: A Survey , 2009, IEEE Transactions on Autonomous Mental Development.

[60]  Jürgen Schmidhuber,et al.  Formal Theory of Creativity, Fun, and Intrinsic Motivation (1990–2010) , 2010, IEEE Transactions on Autonomous Mental Development.

[61]  Olivier Sigaud,et al.  Autonomous online learning of velocity kinematics on the iCub: A comparative study , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[62]  Giulio Sandini,et al.  An experimental evaluation of a novel minimum-jerk cartesian controller for humanoid robots , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[63]  Pierre-Yves Oudeyer,et al.  What is Intrinsic Motivation? A Typology of Computational Approaches , 2007, Frontiers Neurorobotics.

[64]  Giulio Sandini,et al.  Force feedback exploiting tactile and proximal force/torque sensing , 2012, Autonomous Robots.

[65]  P. L. Adams THE ORIGINS OF INTELLIGENCE IN CHILDREN , 1976 .

[66]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[67]  David Vernon,et al.  A Roadmap for Cognitive Development in Humanoid Robots , 2011, Cognitive Systems Monographs.

[68]  David Filliat,et al.  Developmental object learning through manipulation and human demonstration , 2013 .

[69]  Charles C. Kemp,et al.  What can I control? A framework for robot self-discovery , 2006 .

[70]  José Santos-Victor,et al.  Learning Sensory-Motor Maps for Redundant Robots , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[71]  Yoshua Bengio,et al.  Greedy Layer-Wise Training of Deep Networks , 2006, NIPS.

[72]  三嶋 博之 The theory of affordances , 2008 .

[73]  Rachid Alami,et al.  > Replace This Line with Your Paper Identification Number (double-click Here 1 , 2022 .

[74]  David A. Cohn,et al.  Active Learning with Statistical Models , 1996, NIPS.

[75]  F. D. Horowitz,et al.  Theories of Developmental Psychology (2nd ed.). , 1989 .

[76]  Jürgen Schmidhuber,et al.  Curious model-building control systems , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[77]  M. Botvinick,et al.  Neural representations of events arise from temporal community structure , 2013, Nature Neuroscience.

[78]  David Filliat,et al.  A visual bag of words method for interactive qualitative localization and mapping , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[79]  Michael Beetz,et al.  ORO, a knowledge management module for cognitive architectures in robotics , 2010, IROS 2010.

[80]  Antoine Cully,et al.  Behavioral repertoire learning in robotics , 2013, GECCO '13.

[81]  R. Shaw,et al.  Perceiving, Acting and Knowing : Toward an Ecological Psychology , 1978 .

[82]  David Filliat,et al.  Improving object learning through manipulation and robot self-identification , 2013, 2013 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[83]  David Filliat,et al.  Developmental approach for interactive object discovery , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[84]  Zoran Popovic,et al.  Compact character controllers , 2009, ACM Trans. Graph..

[85]  E. Gibson Exploratory behavior in the development of perceiving, acting, and the acquiring of knowledge. , 1988 .

[86]  Jochen J. Steil,et al.  Reaching movement generation with a recurrent neural network based on learning inverse kinematics for the humanoid robot iCub , 2009, 2009 9th IEEE-RAS International Conference on Humanoid Robots.

[87]  Emre Ugur,et al.  Traversability: A Case Study for Learning and Perceiving Affordances in Robots , 2010, Adapt. Behav..

[88]  Moritz Tenorth,et al.  KnowRob: A knowledge processing infrastructure for cognition-enabled robots , 2013, Int. J. Robotics Res..

[89]  Benjamin Schrauwen,et al.  Towards Learning Inverse Kinematics with a Neural Network Based Tracking Controller , 2011, ICONIP.

[90]  Edward L. Deci,et al.  Intrinsic Motivation and Self-Determination in Human Behavior , 1975, Perspectives in Social Psychology.

[91]  Sinan Kalkan,et al.  Learning Adjectives and Nouns from Affordances on the iCub Humanoid Robot , 2012, SAB.

[92]  Daniel C. Richardson,et al.  Infants learn about objects from statistics and people. , 2011, Developmental psychology.

[93]  Andrew McCallum,et al.  Toward Optimal Active Learning through Sampling Estimation of Error Reduction , 2001, ICML.

[94]  Z. Pylyshyn Visual indexes, preconceptual objects, and situated vision , 2001, Cognition.

[95]  Mohamed Chetouani,et al.  Perception and human interaction for developmental learning of objects and affordances , 2012, 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012).

[96]  Peter Ford Dominey,et al.  The Coordinating Role of Language in Real-Time Multimodal Learning of Cooperative Tasks , 2013, IEEE Transactions on Autonomous Mental Development.

[97]  Pierre-Yves Oudeyer,et al.  Guest Editorial Active Learning and Intrinsically Motivated Exploration in Robots: Advances and Challenges , 2010, IEEE Trans. Auton. Ment. Dev..

[98]  Giorgio Metta,et al.  Towards long-lived robot genes , 2008, Robotics Auton. Syst..

[99]  James L. McClelland,et al.  Autonomous Mental Development by Robots and Animals , 2001, Science.

[100]  P. H. Miller Theories of developmental psychology , 1983 .

[101]  Giulio Sandini,et al.  Stochastic optimal control with variable impedance manipulators in presence of uncertainties and delayed feedback , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[102]  Giulio Sandini,et al.  In Press, Ieee Transactions on Autonomous Mental Development , 2010 .

[103]  Geoffrey E. Hinton Learning multiple layers of representation , 2007, Trends in Cognitive Sciences.