Reaching in clutter with whole-arm tactile sensing

Clutter creates challenges for robot manipulation, including a lack of non-contact trajectories and reduced visibility for line-of-sight sensors. We demonstrate that robots can use whole-arm tactile sensing to perceive clutter and maneuver within it, while keeping contact forces low. We first present our approach to manipulation, which emphasizes the benefits of making contact across the entire manipulator and assumes the manipulator has low-stiffness actuation and tactile sensing across its entire surface. We then present a novel controller that exploits these assumptions. The controller only requires haptic sensing, handles multiple contacts, and does not need an explicit model of the environment prior to contact. It uses model predictive control with a time horizon of length one and a linear quasi-static mechanical model. In our experiments, the controller enabled a real robot and a simulated robot to reach goal locations in a variety of environments, including artificial foliage, a cinder block, and randomly generated clutter, while keeping contact forces low. While reaching, the robots performed maneuvers that included bending objects, compressing objects, sliding objects, and pivoting around objects. In simulation, whole-arm tactile sensing also outperformed per-link force-torque sensing in moderate clutter, with the relative benefits increasing with the amount of clutter.

[1]  G. Dantzig,et al.  COMPLEMENTARY PIVOT THEORY OF MATHEMATICAL PROGRAMMING , 1968 .

[2]  Herbert A. Simon,et al.  The Sciences of the Artificial , 1970 .

[3]  Ron S. Kenett,et al.  Statistics for Business and Economics. , 1988 .

[4]  John J. Craig,et al.  Hybrid position/force control of manipulators , 1981 .

[5]  Daniel E. Whitney,et al.  Quasi-Static Assembly of Compliantly Supported Rigid Parts , 1982 .

[6]  John Kenneth Salisbury,et al.  Interpretation of contact geometries from force measurements , 1984, ICRA.

[7]  N. Hogan Adaptive control of mechanical impedance by coactivation of antagonist muscles , 1984 .

[8]  K. Johnson Contact Mechanics: Frontmatter , 1985 .

[9]  K. Johnson Contact Mechanics: Normal contact of elastic solids – Hertz theory , 1985 .

[10]  Tomas Lozano-Perez A simple motion-planning algorithm for general robot manipulators , 1986, IEEE J. Robotics Autom..

[11]  Oussama Khatib,et al.  A unified approach for motion and force control of robot manipulators: The operational space formulation , 1987, IEEE J. Robotics Autom..

[12]  D. Sims,et al.  Risk Factors in the Diabetic Foot , 1988 .

[13]  Neville Hogan,et al.  On the stability of manipulators performing contact tasks , 1988, IEEE J. Robotics Autom..

[14]  Robert Sandy,et al.  Statistics for Business and Economics , 1989 .

[15]  Brian S. Eberman,et al.  Whole-arm manipulation : kinematics and control , 1989 .

[16]  Manfred Morari,et al.  Model predictive control: Theory and practice - A survey , 1989, Autom..

[17]  Norman I. Badler,et al.  Anthropometry for Computer Animated Human Figures , 1989 .

[18]  Daniel Thalmann,et al.  State-of-the-art in Computer Animation , 1989, Springer Japan.

[19]  John Kenneth Salisbury,et al.  Determination of Manipulator Contact Information from Joint Torque Measurements , 1989, ISER.

[20]  R. McN. Alexander,et al.  Three Uses for Springs in Legged Locomotion , 1990, Int. J. Robotics Res..

[21]  Richard W. Cottle,et al.  Linear Complementarity Problem. , 1992 .

[22]  R. Shadmehr,et al.  Control of Equilibrium Position and Stiffness Through Postural Modules. , 1993, Journal of motor behavior.

[23]  Antonio Bicchi,et al.  Force distribution in multiple whole-limb manipulation , 1993, [1993] Proceedings IEEE International Conference on Robotics and Automation.

[24]  John Kenneth Salisbury,et al.  Contact Sensing from Force Measurements , 1990, Int. J. Robotics Res..

[25]  S. LaValle,et al.  Motion Planning , 2008, Springer Handbook of Robotics.

[26]  Kazuo Tanie,et al.  Contact point detection for grasping an unknown object using self-posture changeability , 1990, IEEE Trans. Robotics Autom..

[27]  Richard M. Murray,et al.  A Mathematical Introduction to Robotic Manipulation , 1994 .

[28]  Pierre E. Dupont,et al.  Jamming and wedging in constrained rigid-body dynamics , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[29]  Donald A. Streit,et al.  Configuration dependent stiffness of the PUMA 560 manipulator: Analytical and experimental results , 1995 .

[30]  Matthew M. Williamson,et al.  Series elastic actuators , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[31]  Matthew M. Williamson,et al.  Postural primitives: Interactive Behavior for a Humanoid Robot Arm , 1996 .

[32]  Pattie Maes,et al.  Postural primitives: Interactive Behavior for a Humanoid Robot Arm , 1996 .

[33]  野間 春生,et al.  Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems 参加報告 , 1997 .

[34]  M. Coleman,et al.  The simplest walking model: stability, complexity, and scaling. , 1998, Journal of biomechanical engineering.

[35]  Jay H. Lee,et al.  Model predictive control: past, present and future , 1999 .

[36]  Allison M. Okamura,et al.  Haptic exploration of fine surface features , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[37]  I. Whishaw,et al.  How skilled are the skilled limb movements of the raccoon (Procyon lotor)? , 1999, Behavioural Brain Research.

[38]  Matthew M. Williamson,et al.  Robot arm control exploiting natural dynamics , 1999 .

[39]  Steven M. LaValle,et al.  Randomized Kinodynamic Planning , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[40]  Joris De Schutter,et al.  Estimating First-Order Geometric Parameters and Monitoring Contact Transitions during Force-Controlled Compliant Motion , 1999, Int. J. Robotics Res..

[41]  J. Barbera,et al.  Contact mechanics , 1999 .

[42]  K. Catania A nose that looks like a hand and acts like an eye: the unusual mechanosensory system of the star-nosed mole , 1999, Journal of Comparative Physiology A.

[43]  David Q. Mayne,et al.  Constrained model predictive control: Stability and optimality , 2000, Autom..

[44]  Vijay Kumar,et al.  Robotic grasping and contact: a review , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[45]  Alvin R. Tilley,et al.  The Measure of Man and Woman: Human Factors in Design , 2001 .

[46]  Allison M. Okamura,et al.  Feature-guided exploration with a robotic finger , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[47]  Chee-Meng Chew,et al.  Virtual Model Control: An Intuitive Approach for Bipedal Locomotion , 2001, Int. J. Robotics Res..

[48]  Daniel E. Koditschek,et al.  RHex: A Simple and Highly Mobile Hexapod Robot , 2001, Int. J. Robotics Res..

[49]  Matthew T. Mason,et al.  Mechanics of Robotic Manipulation , 2001 .

[50]  Jonathan P. How,et al.  Receding horizon control of autonomous aerial vehicles , 2002, Proceedings of the 2002 American Control Conference (IEEE Cat. No.CH37301).

[51]  Gill Andrews Pratt Low Impedance Walking Robots1 , 2002, Integrative and comparative biology.

[52]  Robert Platt,et al.  Extending fingertip grasping to whole body grasping , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[53]  Alin Albu-Schäffer,et al.  Cartesian impedance control of redundant robots: recent results with the DLR-light-weight-arms , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[54]  Hiromi T. Tanaka,et al.  A vision-based haptic exploration , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[55]  N. Hogan,et al.  Impedance and Interaction Control , 2018 .

[56]  N. Dominy Fruits, Fingers, and Fermentation: The Sensory Cues Available to Foraging Primates1 , 2004, Integrative and comparative biology.

[57]  Alessandro De Luca,et al.  An adapt-and-detect actuator FDI scheme for robot manipulators , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[58]  Oussama Khatib,et al.  Synthesis of Whole-Body Behaviors through Hierarchical Control of Behavioral Primitives , 2005, Int. J. Humanoid Robotics.

[59]  Stephen P. DeWeerth,et al.  Biologically Inspired Joint Stiffness Control , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[60]  Timothy Bretl,et al.  Non-gaited humanoid locomotion planning , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[61]  Stephen Paul Buerger,et al.  Stable, high-force, low-impedance robotic actuators for human-interactive machines , 2005 .

[62]  L. Natale,et al.  A Sensitive Approach to Grasping , 2005 .

[63]  Nancy M. Amato,et al.  Planning motion in completely deformable environments , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[64]  Dana H. Ballard,et al.  An Equilibrium Point based Model Unifying Movement Control in Humanoids , 2006, Robotics: Science and Systems.

[65]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[66]  Tomás Lozano-Pérez,et al.  Imitation Learning of Whole-Body Grasps , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[67]  Alessandro De Luca,et al.  Collision Detection and Safe Reaction with the DLR-III Lightweight Manipulator Arm , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[68]  Pierre-Brice Wieber,et al.  Trajectory Free Linear Model Predictive Control for Stable Walking in the Presence of Strong Perturbations , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[69]  M. Hersch,et al.  A biologically-inspired controller for reaching movements , 2006, The First IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, 2006. BioRob 2006..

[70]  Christoph Borst,et al.  Autonomous opening of a door with a mobile manipulator: A case study , 2007 .

[71]  Stephen P. Buerger,et al.  Complementary Stability and Loop Shaping for Improved Human–Robot Interaction , 2007, IEEE Transactions on Robotics.

[72]  Charles C. Kemp,et al.  Human-Robot Interaction for Cooperative Manipulation: Handing Objects to One Another , 2007, RO-MAN 2007 - The 16th IEEE International Symposium on Robot and Human Interactive Communication.

[73]  Satoshi Kagami,et al.  Planning and executing navigation among movable obstacles , 2007, Adv. Robotics.

[74]  C.M. Gosselin,et al.  Computationally Efficient Predictive Robot Control , 2007, IEEE/ASME Transactions on Mechatronics.

[75]  S. LaValle,et al.  Chapter 5 Motion Planning , 2007 .

[76]  L. Bianchi Mechanotransduction: Touch and Feel at the Molecular Level as Modeled in Caenorhabditis elegans , 2007, Molecular Neurobiology.

[77]  山田 祐,et al.  Open Dynamics Engine を用いたスノーボードロボットシミュレータの開発 , 2007 .

[78]  Charles C. Kemp,et al.  Two Arms Are Better Than One: A Behavior Based Control System for Assistive Bimanual Manipulation , 2007 .

[79]  Tamim Asfour,et al.  Manipulation Planning Among Movable Obstacles , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[80]  Oussama Khatib,et al.  Probabilistic Estimation of Whole Body Contacts for Multi-Contact Robot Control , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[81]  Alessandro De Luca,et al.  Collision detection and reaction: A contribution to safe physical Human-Robot Interaction , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[82]  Katie Byl,et al.  Approximate optimal control of the compass gait on rough terrain , 2008, 2008 IEEE International Conference on Robotics and Automation.

[83]  Joel W. Burdick,et al.  Contact Modeling and Manipulation , 2008, Springer Handbook of Robotics.

[84]  Kevin Blankespoor,et al.  BigDog, the Rough-Terrain Quadruped Robot , 2008 .

[85]  Ashutosh Saxena,et al.  Robotic Grasping of Novel Objects using Vision , 2008, Int. J. Robotics Res..

[86]  Howie Choset,et al.  Toroidal skin drive for snake robot locomotion , 2008, 2008 IEEE International Conference on Robotics and Automation.

[87]  James J. Kuffner,et al.  OpenRAVE: A Planning Architecture for Autonomous Robotics , 2008 .

[88]  Oussama Khatib,et al.  Robot multiple contact control , 2008, Robotica.

[89]  Stephen P. DeWeerth,et al.  The role of passive joint stiffness and active knee control in robotic leg swinging: applications to dynamic walking , 2008 .

[90]  Advait Jain,et al.  Behavior-Based Door Opening with Equilibrium Point Control , 2009 .

[91]  J. Randall Flanagan,et al.  Coding and use of tactile signals from the fingertips in object manipulation tasks , 2009, Nature Reviews Neuroscience.

[92]  Henrik Gordon Petersen,et al.  Tactile object exploration using cursor navigation sensors , 2009, World Haptics 2009 - Third Joint EuroHaptics conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[93]  Michael Beetz,et al.  Compact models of human reaching motions for robotic control in everyday manipulation tasks , 2009, 2009 IEEE 8th International Conference on Development and Learning.

[94]  Advait Jain,et al.  Pulling open novel doors and drawers with equilibrium point control , 2009, 2009 9th IEEE-RAS International Conference on Humanoid Robots.

[95]  Abderrahmane Kheddar,et al.  Contact planning for acyclic motion with tasks constraints , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[96]  Tamim Asfour,et al.  Dynamic Potential Fields for Dexterous Tactile Exploration , 2009, Human Centered Robot Systems, Cognition, Interaction, Technology.

[97]  R. Klatzky,et al.  Haptic perception: A tutorial , 2009, Attention, perception & psychophysics.

[98]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[99]  Eiichi Yoshida,et al.  Generation of dynamic multi-contact motions: 2D case studies , 2010, 2010 10th IEEE-RAS International Conference on Humanoid Robots.

[100]  Matei T. Ciocarlie,et al.  Contact-reactive grasping of objects with partial shape information , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[101]  Robert W. Platt,et al.  Using Bayesian filtering to interpret tactile data during flexible materials manipulation , 2010 .

[102]  Advait Jain,et al.  Pulling open doors and drawers: Coordinating an omni-directional base and a compliant arm with Equilibrium Point control , 2010, 2010 IEEE International Conference on Robotics and Automation.

[103]  Sean R. Anderson,et al.  Towards biomimetic vibrissal tactile sensing for robot exploration, navigation, and object recognition in hazardous environments , 2010 .

[104]  Pieter Abbeel,et al.  Autonomous Helicopter Aerobatics through Apprenticeship Learning , 2010, Int. J. Robotics Res..

[105]  Giulio Sandini,et al.  Approximate optimal control for reaching and trajectory planning in a humanoid robot , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[106]  Wolfram Burgard,et al.  Robotics: Science and Systems XV , 2010 .

[107]  Geoffrey A. Hollinger,et al.  HERB: a home exploring robotic butler , 2010, Auton. Robots.

[108]  Oussama Khatib,et al.  Compliant Control of Multicontact and Center-of-Mass Behaviors in Humanoid Robots , 2010, IEEE Transactions on Robotics.

[109]  Yang Ding,et al.  Biophysically inspired development of a sand-swimming robot , 2010, Robotics: Science and Systems.

[110]  Siddhartha S. Srinivasa,et al.  Proprioceptive Localization for Mobile Manipulators , 2010 .

[111]  Leslie Pack Kaelbling,et al.  Task-Driven Tactile Exploration , 2010, Robotics: Science and Systems.

[112]  Charles C. Kemp,et al.  Visual odometry and control for an omnidirectional mobile robot with a downward-facing camera , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[113]  Kara L. Marshall,et al.  The cell biology of touch , 2010, The Journal of cell biology.

[114]  Advait Jain,et al.  EL-E: an assistive mobile manipulator that autonomously fetches objects from flat surfaces , 2010, Auton. Robots.

[115]  Oussama Khatib,et al.  Global Localization of Objects via Touch , 2011, IEEE Transactions on Robotics.

[116]  Jean-Pierre Merlet,et al.  Interval-Analysis-Based Determination of the Wrench-Feasible Workspace of Parallel Cable-Driven Robots , 2011, IEEE Transactions on Robotics.

[117]  Emanuel Todorov,et al.  First-exit model predictive control of fast discontinuous dynamics: Application to ball bouncing , 2011, 2011 IEEE International Conference on Robotics and Automation.

[118]  Jan Tommy Gravdahl,et al.  Motion planning and control of robotic manipulators on seaborne platforms , 2011 .

[119]  Ron Alterovitz,et al.  Motion Planning Under Uncertainty In Highly Deformable Environments , 2011, Robotics: Science and Systems.

[120]  Sachin Chitta,et al.  Human-Inspired Robotic Grasp Control With Tactile Sensing , 2011, IEEE Transactions on Robotics.

[121]  S. Haddadin,et al.  Dynamic Motion Planning for Robots in Partially Unknown Environments , 2011 .

[122]  Wolfram Burgard,et al.  Using Gaussian Process Regression for Efficient Motion Planning in Environments with Deformable Objects , 2011, Automated Action Planning for Autonomous Mobile Robots.

[123]  Yuval Tassa,et al.  Infinite-Horizon Model Predictive Control for Periodic Tasks with Contacts , 2011, Robotics: Science and Systems.

[124]  Siddhartha S. Srinivasa,et al.  A Framework for Push-Grasping in Clutter , 2011, Robotics: Science and Systems.

[125]  Stefan Schaal,et al.  Online movement adaptation based on previous sensor experiences , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[126]  Robert Platt,et al.  Using Bayesian Filtering to Localize Flexible Materials During Manipulation , 2011, IEEE Transactions on Robotics.

[127]  Wolfram Burgard,et al.  Tactile Sensing for Mobile Manipulation , 2011, IEEE Transactions on Robotics.

[128]  Ian R. Manchester,et al.  Stable dynamic walking over uneven terrain , 2011, Int. J. Robotics Res..

[129]  Giulio Sandini,et al.  Force Control and Reaching Movements on the iCub Humanoid Robot , 2011, ISRR.

[130]  Gregory D. Hager,et al.  Tactile-Object Recognition From Appearance Information , 2011, IEEE Transactions on Robotics.

[131]  Alin Albu-Schäffer,et al.  A versatile biomimetic controller for contact tooling and haptic exploration , 2012, 2012 IEEE International Conference on Robotics and Automation.

[132]  Siddhartha S. Srinivasa,et al.  Autonomous manipulation with a general-purpose simple hand , 2011, Int. J. Robotics Res..

[133]  Nicholas Roy,et al.  A Framework for Push-Grasping in Clutter , 2012 .

[134]  Leila Takayama,et al.  Strategies for human-in-the-loop robotic grasping , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[135]  Mark R. Cutkosky,et al.  Dynamic Tactile Sensing , 2014, The Human Hand as an Inspiration for Robot Hand Development.