Robot manipulation in human environments

Human environments present special challenges for robot manipulation. They are often dynamic, difficult to predict, and beyond the control of a robot engineer. Fortunately, many characteristics of these settings can be used to a robot's advantage. Human environments are typically populated by people, and a robot can rely on the guidance and assistance of a human collaborator. Everyday objects exhibit common, task-relevant features that reduce the cognitive load required for the object's use. Many tasks can be achieved through the detection and control of these sparse perceptual features. And finally, a robot is more than a passive observer of the world. It can use its body to reduce its perceptual uncertainty about the world. In this thesis we present advances in robot manipulation that address the unique challenges of human environments. We describe the design of a humanoid robot named Domo, develop methods that allow Domo to assist a person in everyday tasks, and discuss general strategies for building robots that work alongside people in their homes and workplaces. (Copies available exclusively from MIT Libraries, Rm. 14-0551, Cambridge, MA 02139-4307. Ph. 617-253-5668; Fax 617-253-1690.)

[1]  C. Kemp,et al.  Toward Robot Learning of Tool Manipulation from Human Demonstration , 2006 .

[2]  Roderic A. Grupen,et al.  User intentions funneled through a human-robot interface , 2005, IUI '05.

[3]  Fumio Kanehiro,et al.  Cooperative works by a human and a humanoid robot , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[4]  Jonathan E. Clark,et al.  Fast and Robust: Hexapedal Robots via Shape Deposition Manufacturing , 2002 .

[5]  Rodney A. Brooks,et al.  Challenges for complete creature architectures , 1991 .

[6]  Michael R. Zinn,et al.  A New Actuation Approach for Human Friendly Robot Design , 2004, Int. J. Robotics Res..

[7]  Sungchul Kang,et al.  Development of a Joint Torque Sensor Fully Integrated with an Actuator , 2005 .

[8]  Rodney A. Brooks L – A Common Lisp for Embedded Systems , 2004 .

[9]  Vladimir J. Lumelsky,et al.  Final report for the DARPA/NSF interdisciplinary study on human-robot interaction , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[10]  J. Edward Colgate,et al.  Controlling the apparent inertia of passive human-interactive robots , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[11]  M. Matarić,et al.  A hands-off physical therapy assistance robot for cardiac patients , 2005, 9th International Conference on Rehabilitation Robotics, 2005. ICORR 2005..

[12]  Matthew M. Williamson,et al.  Exploiting Natural Dynamics in Robot Control , 1998 .

[13]  John Kenneth Salisbury,et al.  Preliminary design of a whole-arm manipulation system (WAMS) , 1988, Proceedings. 1988 IEEE International Conference on Robotics and Automation.

[14]  Cynthia Breazeal,et al.  Social interactions in HRI: the robot view , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[15]  William Townsend,et al.  Teleoperator slave ‐ WAM design methodology , 1999 .

[16]  Jessica K. Hodgins,et al.  Generalizing Demonstrated Manipulation Tasks , 2002, WAFR.

[17]  Paul Fitzpatrick,et al.  Exploiting cross-modal rhythm for robot perception of objects , 2004 .

[18]  Charles C. Kemp,et al.  A wearable system that learns a kinematic model and finds structure in everyday manipulation by using absolute orientation sensors and a camera , 2005 .

[19]  Ashutosh Saxena,et al.  Learning to Grasp Novel Objects Using Vision , 2006, ISER.

[20]  Norman I. Badler,et al.  Real-Time Inverse Kinematics Techniques for Anthropomorphic Limbs , 2000, Graph. Model..

[21]  Mitul Saha,et al.  Motion planning for robotic manipulation of deformable linear objects , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[22]  Chris Lovchik,et al.  The Robonaut hand: a dexterous robot hand for space , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[23]  Rodney A. Brooks,et al.  Sensing and Manipulating Built-for-Human Environments , 2004, Int. J. Humanoid Robotics.

[24]  John Kenneth Salisbury,et al.  Playing it safe [human-friendly robots] , 2004, IEEE Robotics & Automation Magazine.

[25]  Karl T. Ulrich,et al.  Intrinsically Safer Robots , 1995 .

[26]  John M. Hollerbach,et al.  The Calibration Index and Taxonomy for Robot Kinematic Calibration Methods , 1996, Int. J. Robotics Res..

[27]  Robert Platt,et al.  Extending fingertip grasping to whole body grasping , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[28]  Eric Jones,et al.  SciPy: Open Source Scientific Tools for Python , 2001 .

[29]  SantiniFabrizio,et al.  Active estimation of distance in a robotic system that replicates human eye movement , 2007 .

[30]  Ronald C. Arkin,et al.  An Behavior-based Robotics , 1998 .

[31]  Giorgio Metta,et al.  Better Vision through Manipulation , 2003, Adapt. Behav..

[32]  Giorgio Metta,et al.  Early integration of vision and manipulation , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[33]  J. Gibson The Ecological Approach to Visual Perception , 1979 .

[34]  Jan Wikander,et al.  Tactile sensing in intelligent robotic manipulation - a review , 2005, Ind. Robot.

[35]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[36]  Ehud Rivlin,et al.  Function From Motion , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Steve Block,et al.  Control and Estimation for Cooperative Manipulator Tasks , 2006 .

[38]  G. Giralt,et al.  Safe and dependable physical human-robot interaction in anthropic domains: State of the art and challenges , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[39]  R. Brooks,et al.  The cog project: building a humanoid robot , 1999 .

[40]  Russell H. Taylor,et al.  Automatic Synthesis of Fine-Motion Strategies for Robots , 1984 .

[41]  Ronald C. Arkin,et al.  Anticipatory robot navigation by simultaneously localizing and building a cognitive map , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[42]  Antonio Torralba,et al.  Modeling global scene factors in attention. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[43]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[44]  Fredrik Rehnmark,et al.  Robonaut: NASA's Space Humanoid , 2000, IEEE Intell. Syst..

[45]  Karun B. Shimoga,et al.  Robot Grasp Synthesis Algorithms: A Survey , 1996, Int. J. Robotics Res..

[46]  Matthew T. Mason,et al.  An exploration of sensorless manipulation , 1986, IEEE J. Robotics Autom..

[47]  E. Torres-Jara Obrero: a platform for sensitive manipulation , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[48]  Volker Graefe,et al.  Design Principles for Dependable Robotic Assistants , 2004, Int. J. Humanoid Robotics.

[49]  Randal C. Nelson,et al.  Visual space task specification, planning and control , 1995, Proceedings of International Symposium on Computer Vision - ISCV.

[50]  Oliver Brock,et al.  A Framework For Humanoid Control and Intelligence , 2003 .

[51]  Peng Pan,et al.  Human interaction with passive assistive robots , 2005, 9th International Conference on Rehabilitation Robotics, 2005. ICORR 2005..

[52]  James S. Albus,et al.  Outline for a theory of intelligence , 1991, IEEE Trans. Syst. Man Cybern..

[53]  Yukie Nagai,et al.  Learning to comprehend deictic gestures in robots and human infants , 2005, ROMAN 2005. IEEE International Workshop on Robot and Human Interactive Communication, 2005..

[54]  Oussama Khatib,et al.  The virtual linkage: a model for internal forces in multi-grasp manipulation , 1993, [1993] Proceedings IEEE International Conference on Robotics and Automation.

[55]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  David G. Stork,et al.  Pattern classification, 2nd Edition , 2000 .

[57]  Giorgio Metta,et al.  Grounding vision through experimental manipulation , 2003, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[58]  Chee-Meng Chew,et al.  Virtual Model Control: An Intuitive Approach for Bipedal Locomotion , 2001, Int. J. Robotics Res..

[59]  Paul Cisek,et al.  Neural representations of motor plans, desired trajectories, and controlled objects , 2005, Cognitive Processing.

[60]  Takashi Maeno,et al.  Five-fingered Robot Hand using Ultrasonic Motors and Elastic Elements , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[61]  Illah R. Nourbakhsh,et al.  A survey of socially interactive robots , 2003, Robotics Auton. Syst..

[62]  Robert O. Ambrose,et al.  A mobile autonomous humanoid assistant , 2004, 4th IEEE/RAS International Conference on Humanoid Robots, 2004..

[63]  Roger Orpwood,et al.  A WHEELCHAIR MOUNTED ASSISTIVE ROBOT , 1999 .

[64]  Antonio Bicchi,et al.  Hands for dexterous manipulation and robust grasping: a difficult road toward simplicity , 2000, IEEE Trans. Robotics Autom..

[65]  Marc H. Raibert,et al.  Legged Robots That Balance , 1986, IEEE Expert.

[66]  Eric Huber,et al.  Using a hybrid of silhouette and range templates for real-time pose estimation , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[67]  Aude Billard,et al.  Teaching a Humanoid Robot to Recognize and Reproduce Social Cues , 2006, ROMAN 2006 - The 15th IEEE International Symposium on Robot and Human Interactive Communication.

[68]  Alin Albu-Schäffer,et al.  DLR's torque-controlled light weight robot III-are we reaching the technological limits now? , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[69]  Stephen Hart,et al.  A Framework for Learning Declarative Structure , 2006 .

[70]  C. Kemp,et al.  Robot Manipulation of Human Tools : Autonomous Detection and Control of Task Relevant Features , 2006 .

[71]  Helge Ritter,et al.  Robot guidance by human pointing gestures , 1996, Proceedings of International Workshop on Neural Networks for Identification, Control, Robotics and Signal/Image Processing.

[72]  Dale A. Lawrence,et al.  Actuator limitations on achievable manipulator impedance , 1989, Proceedings, 1989 International Conference on Robotics and Automation.

[73]  Lorenzo Natale,et al.  Tapping into Touch , 2005 .

[74]  Justus H. Piater,et al.  Developing haptic and visual perceptual categories for reaching and grasping with a humanoid robot , 2001, Robotics Auton. Syst..

[75]  G. Honderd,et al.  Rehabilitation robotics: the MANUS concept , 1991, Fifth International Conference on Advanced Robotics 'Robots in Unstructured Environments.

[76]  Michael R. Lowry,et al.  Learning Physical Descriptions From Functional Definitions, Examples, and Precedents , 1983, AAAI.

[77]  Lijin Aryananda Attending to Learn and Learning to Attend for a , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[78]  Jeff Weber,et al.  MERTZ: a quest for a robust and scalable active vision humanoid head robot , 2004, 4th IEEE/RAS International Conference on Humanoid Robots, 2004..

[79]  Samuel R. Buss,et al.  Selectively Damped Least Squares for Inverse Kinematics , 2005, J. Graph. Tools.

[80]  Ian Horswill,et al.  Polly: A Vision-Based Artificial Agent , 1993, AAAI.

[81]  Danica Kragic,et al.  Cue integration for visual servoing , 2001, IEEE Trans. Robotics Autom..

[82]  V. Braitenberg Vehicles, Experiments in Synthetic Psychology , 1984 .

[83]  Roderic A. Grupen,et al.  2-D contact detection and localization using proprioceptive information , 1994, IEEE Trans. Robotics Autom..

[84]  Gordon Cheng,et al.  Online learning of a motor map for humanoid robot reaching , 2003 .

[85]  R. A. Brooks,et al.  Intelligence without Representation , 1991, Artif. Intell..

[86]  Danica Kragic,et al.  Survey on Visual Servoing for Manipulation , 2002 .

[87]  Robert Platt,et al.  Manipulation gaits: sequences of grasp control tasks , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[88]  Andrea Lockerd Thomaz,et al.  Building an autonomous humanoid tool user , 2004, 4th IEEE/RAS International Conference on Humanoid Robots, 2004..

[89]  C. Breazeal Sociable Machines: Expressive Social Ex-change Between Humans and Robots , 2000 .

[90]  Giulio Sandini,et al.  Learning about objects through action - initial steps towards artificial cognition , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[91]  Maja J. Matarić,et al.  Towards Socially Assistive Robotics , 2006 .

[92]  Maja J. Matarić,et al.  Behavior-Based Control: Main Properties and Implications , 1992 .

[93]  Roderic A. Grupen,et al.  Robust finger gaits from closed-loop controllers , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[94]  Lorenzo Natale,et al.  Linking Action to Perception in a Humanoid Robot: a Developmental Approach to Grasping , 2004 .

[95]  David W. Robinson,et al.  Design and analysis of series elasticity in closed-loop actuator force control , 2000 .

[96]  Brian Scassellati,et al.  Motion-based robotic self-recognition , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[97]  Kathleen E Krichbaum,et al.  Automation as caregiver , 2001 .

[98]  Paul M. Fitzpatrick,et al.  Developmentally deep perceptual system for a humanoid robot , 2003 .

[99]  B. Habibi,et al.  Pengi : An Implementation of A Theory of Activity , 1998 .

[100]  Thierry Siméon,et al.  Manipulation Planning with Probabilistic Roadmaps , 2004, Int. J. Robotics Res..

[101]  S. Buss Introduction to Inverse Kinematics with Jacobian Transpose , Pseudoinverse and Damped Least Squares methods , 2004 .

[102]  H. Yanco,et al.  Automation as Caregiver: A Survey of Issues and Technologies , 2003 .

[103]  Luca Bogoni Functional features for chopping extracted from observations and interactions , 1998, Image Vis. Comput..

[104]  Tamim Asfour,et al.  Programming by demonstration: dual-arm manipulation tasks for humanoid robots , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[105]  William Harwin,et al.  Devices for assisting manipulation: a summary of user task priorities , 1994 .

[106]  Claudio Melchiorri,et al.  Modelling and Controlling the Compliance of a Robotic Hand with Soft Finger-pads , 2005, Multi-point Interaction with Real and Virtual Objects.

[107]  Fumiya Iida,et al.  Sensing through body dynamics , 2006, Robotics Auton. Syst..

[108]  Alessandro Saffiotti,et al.  A hierarchical behavior-based approach to manipulation tasks , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[109]  J. Salisbury,et al.  Active stiffness control of a manipulator in cartesian coordinates , 1980, 1980 19th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[110]  Matthew M. Williamson,et al.  Robot arm control exploiting natural dynamics , 1999 .

[111]  Haruhisa Kawasaki,et al.  Dexterous anthropomorphic robot hand with distributed tactile sensor: Gifu hand II , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[112]  Ronald C. Arkin Homeostatic control for a mobile robot: Dynamic replanning in hazardous environments , 1992, J. Field Robotics.

[113]  Eduardo Torres-Jara,et al.  The power of the dark side: using cast shadows for visually-guided touching , 2004, 4th IEEE/RAS International Conference on Humanoid Robots, 2004..

[114]  Phillip J. McKerrow,et al.  Introduction to robotics , 1991 .

[115]  Andrew W. Fitzgibbon,et al.  Automatic 3D Model Construction for Turn-Table Sequences , 1998, SMILE.

[116]  Justus Piater,et al.  Learning Appearance Features to Support Robotic Manipulation , 2002 .

[117]  J. Wolfe,et al.  Guided Search 2.0 A revised model of visual search , 1994, Psychonomic bulletin & review.

[118]  Rodney A. Brooks,et al.  A Robust Layered Control Syste For A Mobile Robot , 2022 .

[119]  James R. Tresilian,et al.  A simple rule of thumb for elegant prehension , 2001, Current Biology.

[120]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[121]  Antonio Bicchi,et al.  Variable Stiffness Actuators for Fast and Safe Motion Control , 2003, ISRR.

[122]  M. Alexander,et al.  Principles of Neural Science , 1981 .

[123]  H. Kozima,et al.  A Robot that Learns to Communicate with Human Caregivers , 2001 .

[124]  Matthew T. Mason,et al.  The mechanics of manipulation , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[125]  Joelle Pineau,et al.  Pearl: A Mobile Robotic Assistant for the Elderly , 2002 .

[126]  Oussama Khatib,et al.  A unified approach for motion and force control of robot manipulators: The operational space formulation , 1987, IEEE J. Robotics Autom..

[127]  Matthew M. Williamson,et al.  Series elastic actuators , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[128]  Joelle Pineau,et al.  Towards robotic assistants in nursing homes: Challenges and results , 2003, Robotics Auton. Syst..

[129]  Rodney A. Brooks,et al.  Intelligence Without Reason , 1991, IJCAI.

[130]  C. Stiller,et al.  Estimating Motion in Image Sequences A tutorial on modeling and computation of 2 D motion , 2022 .

[131]  O. Jenkins,et al.  Uncovering Success in Manipulation , 2006 .

[132]  M. Rucci,et al.  Contributions of fixational eye movements to the discrimination of briefly presented stimuli. , 2003, Journal of vision.

[133]  Andrea Lockerd Thomaz,et al.  Working collaboratively with humanoid robots , 2004, 4th IEEE/RAS International Conference on Humanoid Robots, 2004..

[134]  H. Aldridge,et al.  Control Architecture for the Robonaut Space Humanoid , 2000 .

[135]  Giorgio Metta,et al.  YARP: Yet Another Robot Platform , 2006 .

[136]  Maurizio Gentilucci,et al.  Grasping an object naturally or with a tool: are these tasks guided by a common motor representation? , 2004, Experimental Brain Research.

[137]  Robert St. Amant,et al.  Tool Use for Autonomous Agents , 2005, AAAI.

[138]  三嶋 博之 The theory of affordances , 2008 .

[139]  G. Revesz,et al.  [Thinking and speaking]. , 1950, Nederlands tijdschrift voor de psychologie en haar grensgebieden.

[140]  Robert O. Ambrose,et al.  Cooperative manipulation between humans and teleoperated agents , 2004, 12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2004. HAPTICS '04. Proceedings..

[141]  Hendrik Van Brussel,et al.  Introducing robots into a human-centred environment: the behaviour-based approach , 2001 .

[142]  Colin Potts,et al.  Design of Everyday Things , 1988 .

[143]  C. Kemp,et al.  What Can I Control ? : The Development of Visual Categories for a Robot ’ s Body and the World that it Influences , 2006 .

[144]  K. Yokoi,et al.  Operating Humanoid Robots in Human Environments , 2006 .

[145]  Sebastian Thrun,et al.  Probabilistic robotics , 2002, CACM.

[146]  K. Kawamura,et al.  The Sensory Ego-Sphere as a Short-Term Memory for Humanoids , 2002 .

[147]  Paul A. Viola,et al.  Robust Real-time Object Detection , 2001 .

[148]  Gordon Cheng,et al.  Continuous humanoid interaction: : An integrated perspective - gaining adaptivity, redundancy, flexibility - in one , 2001, Robotics Auton. Syst..

[149]  K. Shirai,et al.  Controlling gaze of humanoid in communication with human , 1998, Proceedings. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems. Innovations in Theory, Practice and Applications (Cat. No.98CH36190).

[150]  Charles C. Kemp,et al.  Visual Tool Tip Detection and Position Estimation for Robotic Manipulation of Unknown Human Tools , 2005 .

[151]  Fumiya Iida,et al.  Morphological Computation: Connecting Body, Brain, and Environment (特集:ロボティクスと神経科学) , 2005 .

[152]  Brian Scassellati,et al.  Active vision for sociable robots , 2001, IEEE Trans. Syst. Man Cybern. Part A.

[153]  Russell L. Tedrake,et al.  Applied optimal control for dynamically stable legged locomotion , 2004 .

[154]  Robert Platt,et al.  Nullspace composition of control laws for grasping , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[155]  Tamim Asfour,et al.  A modular and distributed embedded control architecture for humanoid robots , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[156]  Alexander H. Waibel,et al.  Natural human-robot interaction using speech, head pose and gestures , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[157]  Michael I. Jordan,et al.  Optimal feedback control as a theory of motor coordination , 2002, Nature Neuroscience.

[158]  Robert D. Howe,et al.  Towards grasping in unstructured environments: grasper compliance and configuration optimization , 2005, Adv. Robotics.

[159]  井上 博允,et al.  Force Feedback in Precise Assembly Tasks , 1975 .

[160]  Imin Kao,et al.  Computing and controlling compliance of a robotic hand , 1989, IEEE Trans. Robotics Autom..

[161]  Matthew T. Mason,et al.  Mechanics of Robotic Manipulation , 2001 .

[162]  O. Brock,et al.  Robots in Human Environments: Basic Autonomous Capabilities , 1999, Int. J. Robotics Res..

[163]  J. Versace A Review of the Severity Index , 1971 .

[164]  Robert Platt,et al.  Learning and generalizing control-based grasping and manipulation skills , 2006 .

[165]  Jonathan H. Connell,et al.  A behavior-based arm controller , 1989, IEEE Trans. Robotics Autom..

[166]  Patrick Doherty,et al.  06231 Executive Summary - Towards Affordance-based Robot Control , 2006, Towards Affordance-Based Robot Control.

[167]  Alexander Stoytchev,et al.  Behavior-Grounded Representation of Tool Affordances , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.