Where can i do this? Geometric Affordances from a Single Example with the Interaction Tensor

This paper introduces and evaluates a new tensor field representation to express the geometric affordance of one object relative to another, a key competence for Cognitive and Autonomous robots. We expand the bisector surface representation to one that is weight-driven and that retains the provenance of surface points with directional vectors. We also incorporate the notion of affordance keypoints which allow for faster decisions at a point of query and with a compact and straightforward descriptor. Using a single interaction example, we are able to generalize to previously-unseen scenarios; both synthetic and also real scenes captured with RGB-D sensors. Evaluations also include crowdsourcing comparisons that confirm the validity of our affordance proposals, which agree on average 84 % of the time with human judgments, that is 20–40 % better than the baseline methods.

[1]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[2]  Gaurav S. Sukhatme,et al.  Semantic labeling of 3D point clouds with object affordance for robot manipulation , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[3]  Alexandre Bernardino,et al.  From human instructions to robot actions: Formulation of goals, affordances and probabilistic planning , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[4]  Li Fei-Fei,et al.  Reasoning about Object Affordances in a Knowledge Base Representation , 2014, ECCV.

[5]  Hedvig Kjellström,et al.  Functional object descriptors for human activity modeling , 2013, 2013 IEEE International Conference on Robotics and Automation.

[6]  Masayuki Inaba,et al.  Determining proper grasp configurations for handovers through observation of object movement patterns and inter-object interactions during usage , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[7]  Sinisa Todorovic,et al.  A Multi-scale CNN for Affordance Segmentation in RGB Images , 2016, ECCV.

[8]  Vladlen Koltun,et al.  Dense scene reconstruction with points of interest , 2013, ACM Trans. Graph..

[9]  Alexandre Bernardino,et al.  Learning at the ends: From hand to tool affordances in humanoid robots , 2017, 2017 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob).

[10]  Nikolaos G. Tsagarakis,et al.  Detecting object affordances with Convolutional Neural Networks , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[11]  Yun Jiang,et al.  Modeling High-Dimensional Humans for Activity Anticipation using Gaussian Process Latent CRFs , 2014, Robotics: Science and Systems.

[12]  Markus Vincze,et al.  Supervised learning of hidden and non-hidden 0-order affordances and detection in real scenes , 2012, 2012 IEEE International Conference on Robotics and Automation.

[13]  Rachid Alami,et al.  Affordance graph: A framework to encode perspective taking and effort based affordances for day-to-day human-robot interaction , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[14]  Andrew Calway,et al.  RGBD relocalisation using pairwise geometry and concise key point sets , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[15]  Taku Komura,et al.  Indexing 3D Scenes Using the Interaction Bisector Surface , 2014, ACM Trans. Graph..

[16]  Rada Mihalcea,et al.  Mining semantic affordances of visual object categories , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Alexandre Bernardino,et al.  Robot anticipation of human intentions through continuous gesture recognition , 2013, 2013 International Conference on Collaboration Technologies and Systems (CTS).

[18]  Giorgio Metta,et al.  Self-supervised learning of grasp dependent tool affordances on the iCub Humanoid robot , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[19]  Yiannis Aloimonos,et al.  Affordance detection of tool parts from geometric features , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[20]  Alexei A. Efros,et al.  From 3D scene geometry to human workspace , 2011, CVPR 2011.

[21]  Lasitha Piyathilaka,et al.  Affordance-map: Mapping human context in 3D scenes using cost-sensitive SVM and virtual human models , 2015, 2015 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[22]  Juergen Gall,et al.  Discovering Object Classes from Activities , 2014, ECCV.

[23]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[24]  Simon J. Julier,et al.  Structured Prediction of Unobserved Voxels from a Single Depth Image , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  三嶋 博之.  The theory of affordances , 2008 .

[26]  Hema Swetha Koppula,et al.  Physically Grounded Spatio-temporal Object Affordances , 2014, ECCV.

[27]  W. H. Warren.  Does This Computational Theory Solve the Right Problem? Marr, Gibson, and the Goal of Vision , 2012, Perception.

[28]  Edwin Olson,et al.  Predicting object functionality using physical simulations , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[29]  Vladlen Koltun,et al.  A Large Dataset of Object Scans , 2016, ArXiv.

[30]  Jin-Hui Zhu,et al.  Affordance Research in Developmental Robotics: A Survey , 2016, IEEE Transactions on Cognitive and Developmental Systems.

[31]  Deva Ramanan,et al.  Predicting Functional Regions on Objects , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[32]  John K. Tsotsos,et al.  Revisiting active perception , 2016, Autonomous Robots.

[33]  Siddhartha S. Srinivasa,et al.  The YCB object and Model set: Towards common benchmarks for manipulation research , 2015, 2015 International Conference on Advanced Robotics (ICAR).