EL-E: an assistive mobile manipulator that autonomously fetches objects from flat surfaces

Assistive mobile robots that autonomously manipulate objects within everyday settings have the potential to improve the lives of the elderly, injured, and disabled. Within this paper, we present the most recent version of the assistive mobile manipulator EL-E with a focus on the subsystem that enables the robot to retrieve objects from and deliver objects to flat surfaces. Once provided with a 3D location via brief illumination with a laser pointer, the robot autonomously approaches the location and then either grasps the nearest object or places an object. We describe our implementation in detail, while highlighting design principles and themes, including the use of specialized behaviors, task-relevant features, and low-dimensional representations.We also present evaluations of EL-E’s performance relative to common forms of variation. We tested EL-E’s ability to approach and grasp objects from the 25 object categories that were ranked most important for robotic retrieval by motor-impaired patients from the Emory ALS Center. Although reliability varied, EL-E succeeded at least once with objects from 21 out of 25 of these categories. EL-E also approached and grasped a cordless telephone on 12 different surfaces including floors, tables, and counter tops with 100% success. The same test using a vitamin pill (ca. 15 mm × 5 mm × 5 mm) resulted in 58% success.

[1]  Heinrich A. Ernst,et al.  MH-1, a computer-operated mechanical hand , 1962, AIEE-IRE '62 (Spring).

[2]  J. C. Rosier,et al.  The MANUS wheelchair-borne manipulator system , 1988 .

[3]  Jakub Segen,et al.  Automatic discovery of robotic grasp configurations , 1988, Proceedings. 1988 IEEE International Conference on Robotics and Automation.

[4]  Jonathan H. Connell,et al.  A behavior-based arm controller , 1989, IEEE Trans. Robotics Autom..

[5]  J. Goodenough,et al.  Perspectives on animal behavior , 1993 .

[6]  William Harwin,et al.  Devices for assisting manipulation: a summary of user task priorities , 1994 .

[7]  H.F.M. Van der Loos,et al.  VA/Stanford rehabilitation robotics research and development program: lessons learned in the application of robotics technology to the field of rehabilitation , 1995 .

[8]  Shimon Edelman,et al.  Learning to grasp using visual information , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[9]  Ronald C. Arkin,et al.  Behavior-based mobile manipulation for drum sampling , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[10]  Jane Smith,et al.  The development of Handy 1, a rehabilitation robotic system to assist the severely disabled , 1998 .

[11]  José Manuel Iñesta Quereda,et al.  Vision-guided grasping of unknown objects for service robots , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[12]  O. Brock,et al.  Robots in Human Environments: Basic Autonomous Capabilities , 1999, Int. J. Robotics Res..

[13]  Paolo Dario,et al.  MOVAID: a personal robot in everyday life of disabled and elderly people , 1999 .

[14]  Oussama Khatib,et al.  ProVAR assistive robot system architecture , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[15]  Eric Jones,et al.  SciPy: Open Source Scientific Tools for Python , 2001 .

[16]  Hendrik Van Brussel,et al.  Introducing robots into a human-centred environment: the behaviour-based approach , 2001 .

[17]  Lars Petersson,et al.  Visually guided manipulation tasks , 2002, Robotics Auton. Syst..

[18]  O. Brock,et al.  Elastic Strips: A Framework for Motion Generation in Human Environments , 2002, Int. J. Robotics Res..

[19]  Christoph Borst,et al.  Towards Service Robots for the Human Environment: the Robutler , 2004 .

[20]  Rolf Dieter Schraft,et al.  Care-O-bot II—Development of a Next Generation Robotic Home Assistant , 2004, Auton. Robots.

[21]  Tamim Asfour,et al.  Programming by demonstration: dual-arm manipulation tasks for humanoid robots , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[22]  Andrea Lockerd Thomaz,et al.  Building an autonomous humanoid tool user , 2004, 4th IEEE/RAS International Conference on Humanoid Robots, 2004..

[23]  Rodney A. Brooks,et al.  Sensing and Manipulating Built-for-Human Environments , 2004, Int. J. Humanoid Robotics.

[24]  Robert Platt,et al.  Re-using schematic grasping policies , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[25]  K. Okada,et al.  Humanoid motion generation system on HRP2-JSK for daily life environment , 2005, IEEE International Conference Mechatronics and Automation, 2005.

[26]  Robert D. Howe,et al.  Towards grasping in unstructured environments: grasper compliance and configuration optimization , 2005, Adv. Robotics.

[27]  L. Natale,et al.  A Sensitive Approach to Grasping , 2005 .

[28]  Oliver Brock,et al.  Humanoid Mobile Manipulation Using Controller Refinement , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[29]  Robert Platt,et al.  Learning and generalizing control-based grasping and manipulation skills , 2006 .

[30]  Hirochika Inoue,et al.  HRP-2W: A humanoid platform for research on support behavior in daily life environments , 2009, Robotics Auton. Syst..

[31]  Sebastian Thrun,et al.  Stanley: The robot that won the DARPA Grand Challenge , 2006, J. Field Robotics.

[32]  N. Cowan,et al.  Task-level control of rapid wall following in the American cockroach , 2006, Journal of Experimental Biology.

[33]  E. Torres-Jara,et al.  Challenges for Robot Manipulation in Human Environments , 2006 .

[34]  Matei T. Ciocarlie,et al.  Dimensionality reduction for hand-independent dexterous robotic grasping , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[35]  Charles C. Kemp,et al.  Challenges for robot manipulation in human environments [Grand Challenges of Robotics] , 2007, IEEE Robotics & Automation Magazine.

[36]  Lena H Ting,et al.  Dimensional reduction in sensorimotor systems: a framework for understanding muscle coordination of posture. , 2007, Progress in brain research.

[37]  Michael A. Goodrich,et al.  Human-Robot Interaction: A Survey , 2008, Found. Trends Hum. Comput. Interact..

[38]  Charles C. Kemp,et al.  Laser pointers and a touch screen: intuitive interfaces for autonomous mobile manipulation for the motor impaired , 2008, Assets '08.

[39]  James J. Little,et al.  Curious George: An attentive semantic robot , 2008, Robotics Auton. Syst..

[40]  Siddhartha S. Srinivasa,et al.  The robotic busboy: Steps towards developing a mobile robotic home assistant , 2008 .

[41]  Katherine M. Tsui,et al.  Development and evaluation of a flexible interface for a wheelchair mounted robotic arm , 2008, 2008 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[42]  Advait Jain,et al.  Behaviors for Robust Door Opening and Doorway Traversal with a Force-Sensing Mobile Manipulator , 2008 .

[43]  Sunil Kumar Agrawal,et al.  Babies driving robots: self-generated mobility in very young infants , 2008, Intell. Serv. Robotics.

[44]  Dov Katz Jacqueline Kenney Oliver Brock How Can Robots Succeed in Unstructured Environments ? , 2008 .

[45]  Lawson L. S. Wong,et al.  Learning Grasp Strategies with Partial Shape Information , 2008, AAAI.

[46]  Odest Chadwicke Jenkins Sparse control for high-DOF assistive robots , 2008, Intell. Serv. Robotics.

[47]  Zhe Xu,et al.  A point-and-click interface for the real world: Laser designation of objects for mobile manipulation , 2008, 2008 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[48]  Ashutosh Saxena,et al.  Robotic Grasping of Novel Objects using Vision , 2008, Int. J. Robotics Res..

[49]  Advait Jain,et al.  A clickable world: Behavior selection through pointing and context for mobile manipulation , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[50]  Nico Blodow,et al.  Towards 3D Point cloud based object maps for household environments , 2008, Robotics Auton. Syst..

[51]  Tamim Asfour,et al.  Toward humanoid manipulation in human-centred environments , 2008, Robotics Auton. Syst..

[52]  H. Nguyen El-E: An Assistive Robot that Fetches Objects from Flat Surfaces , 2008 .

[53]  Christophe Leroux,et al.  SAM: A robotic butler for handicapped people , 2008, RO-MAN 2008 - The 17th IEEE International Symposium on Robot and Human Interactive Communication.

[54]  Tiffany L. Chen,et al.  Hand it over or set it down: A user study of object delivery with an assistive mobile manipulator , 2009, RO-MAN 2009 - The 18th IEEE International Symposium on Robot and Human Interactive Communication.

[55]  Charles C. Kemp,et al.  A list of household objects for robotic retrieval prioritized by people with ALS , 2008, 2009 IEEE International Conference on Rehabilitation Robotics.

[56]  William Whittaker,et al.  Autonomous driving in urban environments: Boss and the Urban Challenge , 2008, J. Field Robotics.

[57]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.