Adapting Everyday Manipulation Skills to Varied Scenarios

We address the problem of executing tool-using manipulation skills in scenarios where the objects to be used may vary. We assume that point clouds of the tool and target object can be obtained, but no interpretation or further knowledge about these objects is provided. The system must interpret the point clouds and decide how to use the tool to complete a manipulation task with a target object; this means it must adjust motion trajectories appropriately to complete the task. We tackle three everyday manipulations: scraping material from a tool into a container, cutting, and scooping from a container. Our solution encodes these manipulation skills in a generic way, with parameters that can be filled in at run-time via queries to a robot perception module; the perception module abstracts the functional parts of the tool and extracts key parameters that are needed for the task. The approach is evaluated in simulation and with selected examples on a PR2 robot.

[1]  Moritz Tenorth,et al.  Knowledge-based Specification of Robot Motions , 2014, ECAI.

[2]  Oliver Kroemer,et al.  Generalizing pouring actions between objects using warped parameters , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[3]  Alois Knoll,et al.  Intuitive instruction of industrial robots: Semantic process descriptions for small lot production , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[4]  Nikolaos G. Tsagarakis,et al.  Detecting object affordances with Convolutional Neural Networks , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[5]  Mehmet R. Dogar,et al.  Physics-Based Manipulation Planning in Cluttered Human Environments , 2013 .

[6]  Frank Guerin,et al.  Learning how a tool affords by simulating 3D models from the web , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[7]  Alois Knoll,et al.  Constraint-based task programming with CAD semantics: From intuitive specification to real-time control , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[8]  Daniel Leidner,et al.  Inferring the effects of wiping motions based on haptic perception , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[9]  Iman Awaad,et al.  Affordance-Based Reasoning in Robot Task Planning , 2013 .

[10]  M. Beetz,et al.  Movement-aware action control — Integrating symbolic and control-theoretic action execution , 2012, 2012 IEEE International Conference on Robotics and Automation.

[11]  Joris De Schutter,et al.  eTaSL/eTC: A constraint-based task specification language and robot controller using expression graphs , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[12]  Paulo Ferreira.  Sampling Superquadric Point Clouds with Normals , 2018, ArXiv.

[13]  Andrea Lockerd Thomaz,et al.  Human-Robot Co-Creativity: Task Transfer on a Spectrum of Similarity , 2017, ICCC.

[14]  Frank Guerin,et al.  Transfer of Tool Affordance and Manipulation Cues with 3D Vision Data , 2017, ArXiv.

[15]  Sonia Chernova,et al.  Towards Robot Adaptability in New Situations , 2015, AAAI Fall Symposia.

[16]  Sanem Sariel,et al.  Cognition-Enabled Robot Manipulation in Human Environments: Requirements, Recent Work, and Open Problems , 2017, IEEE Robotics & Automation Magazine.

[17]  Eren Erdal Aksoy,et al.  Using structural bootstrapping for object substitution in robotic executions of human-like manipulation tasks , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[18]  Daniel Sebastian Leidner,et al.  Cognitive Reasoning for Compliant Robot Manipulation , 2018, Springer Tracts in Advanced Robotics.

[19]  Yiannis Aloimonos,et al.  Affordance detection of tool parts from geometric features , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[20]  Michael Beetz,et al.  Learning models for constraint-based motion parameterization from interactive physics-based simulation , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[21]  Dieter Fox,et al.  Manipulator and object tracking for in-hand 3D object modeling , 2011, Int. J. Robotics Res..

[22]  Rainer Jäkel,et al.  Learning of Generalized Manipulation Strategies in Service Robotics , 2013 .

[23]  Joris De Schutter,et al.  iTASC: a tool for multi-sensor integration in robot manipulation , 2008, 2008 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems.

[24]  Xinyu Liu,et al.  Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics , 2017, Robotics: Science and Systems.

[25]  Moritz Tenorth,et al.  Decomposing CAD models of objects of daily use and reasoning about their functional parts , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[26]  Connor Schenck,et al.  Visual closed-loop control for pouring liquids , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).