Object-object interaction affordance learning

This paper presents a novel object-object affordance learning approach that enables intelligent robots to learn the interactive functionalities of objects from human demonstrations in everyday environments. Instead of considering a single object, we model the interactive motions between paired objects in a human-object-object way. The innate interaction-affordance knowledge of the paired objects are learned from a labeled training dataset that contains a set of relative motions of the paired objects, human actions, and object labels. The learned knowledge is represented with a Bayesian Network, and the network can be used to improve the recognition reliability of both objects and human actions and to generate proper manipulation motion for a robot if a pair of objects is recognized. This paper also presents an image-based visual servoing approach that uses the learned motion features of the affordance in interaction as the control goals to control a robot to perform manipulation tasks.

[1]  T. Ziemke,et al.  Theories and computational models of affordance and mirror systems: An integrative review , 2013, Neuroscience & Biobehavioral Reviews.

[2]  Michael A. Arbib,et al.  Mirror neurons and imitation: A computationally guided review , 2006, Neural Networks.

[3]  L. Wheaton,et al.  One hand, two objects: Emergence of affordance in contexts , 2012, Brain and Cognition.

[4]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[5]  Steven A. Jax,et al.  Response interference between functional and structural actions linked to the same familiar object , 2010, Cognition.

[6]  V. Gallese.  Action representaion and the inferior parietal lobule , 2000 .

[7]  Peter I. Corke,et al.  A tutorial on visual servo control , 1996, IEEE Trans. Robotics Autom..

[8]  Fei-Fei Li,et al.  Modeling mutual context of object and human pose in human-object interaction activities , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Alan F. Smeaton,et al.  Detector adaptation by maximising agreement between independent data sources , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  Larry S. Davis,et al.  Objects in Action: An Approach for Combining Action Understanding and Object Perception , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Danica Kragic,et al.  Visual object-action recognition: Inferring object affordances from human demonstration , 2011, Comput. Vis. Image Underst..

[13]  三嶋 博之.  The theory of affordances , 2008 .

[14]  Rs Roel Pieters,et al.  Visual Servo Control , 2012 .

[15]  Afdc Hamilton,et al.  The motor hierarchy: from kinematics to goals and intentions , 2007 .

[16]  François Chaumette,et al.  Visual servo control. I. Basic approaches , 2006, IEEE Robotics & Automation Magazine.

[17]  Yu Sun,et al.  Learning grasping force from demonstration , 2012, 2012 IEEE International Conference on Robotics and Automation.

[18]  David A. McAllester,et al.  A discriminatively trained, multiscale, deformable part model , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Eren Erdal Aksoy,et al.  Categorizing object-action relations from semantic scene graphs , 2010, 2010 IEEE International Conference on Robotics and Automation.

[20]  Bernt Schiele,et al.  Functional Object Class Detection Based on Learned Affordance Cues , 2008, ICVS.

[21]  A. Hasman,et al.  Probabilistic reasoning in intelligent systems: Networks of plausible inference , 1991 .

[22]  Jiri Matas,et al.  P-N learning: Bootstrapping binary classifiers by structural constraints , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  J. Mazziotta,et al.  Grasping the Intentions of Others with One's Own Mirror Neuron System , 2005, PLoS biology.

[24]  Scott Kuindersma,et al.  Robot learning from demonstration by constructing skill trees , 2012, Int. J. Robotics Res..

[25]  Andrew Zisserman,et al.  Upper Body Detection and Tracking in Extended Signing Sequences , 2011, International Journal of Computer Vision.

[26]  Stefan Schaal,et al.  http://www.jstor.org/about/terms.html. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained , 2007 .

[27]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[28]  G. Rizzolatti,et al.  Parietal Lobe: From Action Organization to Intention Understanding , 2005, Science.

[29]  G. Rizzolatti,et al.  Understanding motor events: a neurophysiological study , 2004, Experimental Brain Research.

[30]  Yun Jiang,et al.  Learning to place new objects in a scene , 2012, Int. J. Robotics Res..

[31]  Aude Billard,et al.  Incremental learning of gestures by imitation in a humanoid robot , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[32]  Yu Sun,et al.  Visual servoing control of a 9-DoF WMRA to perform ADL tasks , 2012, 2012 IEEE International Conference on Robotics and Automation.

[33]  Larry S. Davis,et al.  Ballistic Hand Movements , 2006, AMDO.

[34]  Manolis I. A. Lourakis,et al.  Real-Time Tracking of Multiple Skin-Colored Objects with a Possibly Moving Camera , 2004, ECCV.

[35]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[36]  G. Rizzolatti,et al.  The mirror-neuron system. , 2004, Annual review of neuroscience.

[37]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[38]  Mike Stilman,et al.  The Motion Grammar: Linguistic Perception, Planning, and Control , 2011, Robotics: Science and Systems.

[39]  Luc Van Gool,et al.  Functional categorization of objects using real-time markerless motion capture , 2011, CVPR 2011.

[40]  Luc Van Gool,et al.  What makes a chair a chair? , 2011, CVPR 2011.

[41]  G. Humphreys,et al.  The paired-object affordance effect. , 2010, Journal of experimental psychology. Human perception and performance.

[42]  G. Rizzolatti,et al.  Mirror neuron: a neurological approach to empathy , 2005 .