Kinesthetic teaching of visuomotor coordination for pointing by the humanoid robot iCub

Pointing at something refers to orienting the hand, the arm, the head or the body in the direction of an object or an event. This skill constitutes a basic communicative ability for cognitive agents like, e.g. humanoid robots. The goal of this study is to show that approximate and, in particular, precise pointing can be learned as a direct mapping from the object's pixel coordinates in the visual field to hand positions or to joint angles. This highly nonlinear mapping defines the pose and orientation of a robot's arm. The study underlines that this is possible without calculating the object's depth and 3D position explicitly since only the direction is required. To this aim, three state-of-the-art neural network paradigms (multilayer perceptron, extreme learning machine and reservoir computing) are evaluated on real world data gathered from the humanoid robot iCub. Training data are interactively generated and recorded from kinesthetic teaching for the case of precise pointing. Successful generalization is verified on the iCub using a laser pointer attached to its hand.

[1]  J. J. Aguilar,et al.  Stereo vision for 3D measurement: accuracy analysis, calibration and industrial applications , 1996 .

[2]  Martin Jägersand,et al.  Towards Learning Robotic Reaching and Pointing: An Uncalibrated Visual Servoing Approach , 2009, 2009 Canadian Conference on Computer and Robot Vision.

[3]  Martin Jägersand,et al.  Global Visual-Motor Estimation for Uncalibrated Visual Servoing , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[4]  François Chaumette,et al.  Visual servo control. I. Basic approaches , 2006, IEEE Robotics & Automation Magazine.

[5]  L. Behera,et al.  A hybrid neural control scheme for visual-motor coordination , 1999 .

[6]  Daniel Svozil,et al.  Introduction to multi-layer feed-forward neural networks , 1997 .

[7]  Helge J. Ritter,et al.  Three-dimensional neural net for learning visuomotor coordination of a robot arm , 1990, IEEE Trans. Neural Networks.

[8]  Helge J. Ritter,et al.  Self-Organizing Feature Maps for Modeling and Control of Robotic Manipulators , 2003, J. Intell. Robotic Syst..

[9]  Giulio Sandini,et al.  An experimental evaluation of a novel minimum-jerk cartesian controller for humanoid robots , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  Giulio Sandini,et al.  The iCub humanoid robot: An open-systems platform for research in cognitive development , 2010, Neural Networks.

[11]  Michael A. Arbib,et al.  The handbook of brain theory and neural networks , 1995, A Bradford book.

[12]  Jan Babič,et al.  Inverse Kinematics of Humanoid-Robot Reaching through Human Visuo-Motor Learning , 2010 .

[13]  Brian Scassellati,et al.  A Fast and Efficient Model for Learning to Reach , 2005, Int. J. Humanoid Robotics.

[14]  Alexandre Pouget,et al.  Computational approaches to sensorimotor transformations , 2000, Nature Neuroscience.

[15]  Luís A. Alexandre,et al.  Reservoir computing for static pattern recognition , 2009, ESANN.

[16]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[17]  Jochen J. Steil,et al.  Recurrence Enhances the Spatial Encoding of Static Inputs in Reservoir Networks , 2010, ICANN.

[18]  Klaus Neumann,et al.  Regularization by Intrinsic Plasticity and Its Synergies with Recurrence for Random Projection Methods , 2012 .

[19]  J. J. Guajardo,et al.  Infants’ understanding of the point gesture as an object-directed action , 2002 .

[20]  Gordon Cheng,et al.  Online learning of a motor map for humanoid robot reaching , 2003 .

[21]  Jochen J. Steil,et al.  Learning visuo-motor coordination for pointing without depth calculation , 2012, ESANN.

[22]  José Santos-Victor,et al.  Visual transformations in gesture imitation: what you see is what you do , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[23]  Giulio Sandini,et al.  Autonomous learning of 3D reaching in a humanoid robot , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[24]  Brian Scassellati,et al.  Active Learning of Joint Attention , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[25]  François Chaumette,et al.  Visual servo control. II. Advanced approaches [Tutorial] , 2007, IEEE Robotics & Automation Magazine.

[26]  Wei Wu,et al.  Boundedness and convergence of batch back-propagation algorithm with penalty for feedforward neural networks , 2012, Neurocomputing.

[27]  Qinyu. Zhu.  Extreme Learning Machine , 2013 .

[28]  Pedro U. Lima,et al.  Tracking objects with generic calibrated sensors: An algorithm based on color and 3D shape features , 2010, Robotics Auton. Syst..

[29]  Pattie Maes,et al.  Self-Taught Visually-Guided Pointing for a Humanoid Robot , 1996 .

[30]  Klaus Neumann,et al.  Batch Intrinsic Plasticity for Extreme Learning Machines , 2011, ICANN.

[31]  James S. Albus,et al.  Outline for a theory of intelligence , 1991, IEEE Trans. Syst. Man Cybern..

[32]  Nicola J. Ferrier,et al.  Direct Mapping of Visual Input to Motor Torques , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[33]  Stefan Ulbrich,et al.  General Robot Kinematics Decomposition Without Intermediate Markers , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[34]  Victoria Southgate,et al.  Infant pointing: communication to cooperate or communication to learn? , 2007, Child development.

[35]  M. Tomasello,et al.  12- and 18-Month-Olds Point to Provide Information for Others , 2006 .

[36]  A. Pellionisz,et al.  Tensorial approach to the geometry of brain function: Cerebellar coordination via a metric tensor , 1980, Neuroscience.

[37]  Jochen Triesch,et al.  A Gradient Rule for the Plasticity of a Neuron's Intrinsic Excitability , 2005, ICANN.

[38]  Elizabeth Bates,et al.  The Acquisition of Performatives Prior to Speech. , 1975 .

[39]  M. Spong,et al.  Robot Modeling and Control , 2005 .

[40]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[41]  Seth Hutchinson,et al.  Visual Servo Control Part I: Basic Approaches , 2006 .

[42]  Masatoshi Ishikawa,et al.  Robotic Catching Using a Direct Mapping from Visual Information to Motor Command , 2003 .

[43]  Giulio Sandini,et al.  Joint torque sensing for the upper-body of the iCub humanoid robot , 2009, 2009 9th IEEE-RAS International Conference on Humanoid Robots.