Constrained Manipulation Planning

Every planning problem in robotics involves constraints. Whether the robot must avoid collision or joint limits, there are always states that are not permissible. Some constraints are straightforward to satisfy while others can be so stringent that feasible states are very difficult to find. What makes planning with constraints challenging is that, for many constraints, it is impossible or impractical to provide the planning algorithm with the allowed states explicitly; it must discover these states as it plans. The goal of this thesis is to develop a framework for representing and exploring feasible states in the context of manipulation planning. Planning for manipulation gives rise to a rich variety of tasks that include constraints on collision-avoidance, torque, balance, closed-chain kinematics, and end-effector pose. While many researchers have developed representations and strategies to plan with a specific constraint, the goal of this thesis is to develop a broad representation of constraints on a robot's configuration and identify general strategies to manage these constraints during the planning process. Some of the most important constraints in manipulation planning are functions of the pose of the manipulator's end-effector, so we devote a large part of this thesis to end-effector placement for grasping and transport tasks. We present an efficient approach to generating paths that uses Task Space Regions (TSRs) to specify manipulation tasks which involve end-effector pose goals and/or path constraints. We show how to use TSRs for path planning using the Constrained BiDirectional RRT (CBiRRT2) algorithm and describe several extensions of the TSR representation. Among them are methods to plan with object pose uncertainty, find optimal base placements, and handle more complex pose constraints by chaining TSRs together. We also explore the problem of automatically generating end-effector pose constraints for grasping tasks and present two grasp synthesis algorithms that can generate lists of grasps in extremely cluttered environments. We then describe how to convert these lists of grasps to TSRs so they can be used with CBiRRT2. We have applied our framework to a wide range of problems for several robots, both in simulation and in the real world. These problems include grasping in cluttered environments, lifting heavy objects, two-armed manipulation, and opening doors, to name a few. These example problems demonstrate our framework's practicality, and our proof of probabilistic completeness gives our approach a theoretical foundation. In addition to the above framework, we have also developed the Constellation algorithm for finding configurations that satisfy multiple stringent constraints where other constraint-satisfaction strategies fail. We also present the GradienT-RRT algorithm for planning with soft constraints, which outperforms the state-of-the-art approach to high-dimensional path planning with costs.

[1]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[2]  Richard Fikes,et al.  STRIPS: A New Approach to the Application of Theorem Proving to Problem Solving , 1971, IJCAI.

[3]  Richard Paul Collins Paul,et al.  Modelling, trajectory calculation and servoing of a computer controlled arm , 1972 .

[4]  Harry G. Barrow,et al.  A Versatile Computer-Controlled Assembly System , 1973, IJCAI.

[5]  Russell H. Taylor,et al.  AL, a programming system for automation. , 1974 .

[6]  Ann Patricia Fothergill,et al.  Inferring the Positions of Bodies from Specified Spatial Relationships , 1974, Artif. Intell..

[7]  Russell H. Taylor,et al.  The synthesis of manipulator control programs from task-level specifications , 1976 .

[8]  H. Hanafusa,et al.  Stable Prehension by a Robot Hand with Elastic Fingers , 1977 .

[9]  Michael A. Wesley,et al.  AUTOPASS: An Automatic Programming System for Computer Controlled Mechanical Assembly , 1977, IBM J. Res. Dev..

[10]  Nils J. Nilsson,et al.  Principles of Artificial Intelligence , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  David E. Orin,et al.  Efficient Dynamic Computer Simulation of Robotic Mechanisms , 1982 .

[12]  Russell H. Taylor,et al.  Automatic Synthesis of Fine-Motion Strategies for Robots , 1984 .

[13]  Mark R. Cutkosky,et al.  Robotic grasping and fine manipulation , 1985 .

[14]  Peter Allan Buhr A programming system , 1985 .

[15]  J. Bobrow,et al.  Time-Optimal Control of Robotic Manipulators Along Specified Paths , 1985 .

[16]  Michael A. Erdmann,et al.  Using Backprojections for Fine Motion Planning with Uncertainty , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[17]  Katsushi Ikeuchi,et al.  Determining Grasp Configurations using Photometric Stereo and the PRISM Binocular Stereo System , 1986 .

[18]  L. Devroye Non-Uniform Random Variate Generation , 1986 .

[19]  Peter Cheeseman,et al.  On the Representation and Estimation of Spatial Uncertainty , 1986 .

[20]  Rodney A. Brooks,et al.  An approach to automatic robot programming , 1986, CSC '86.

[21]  W. Eric L. Grimson,et al.  Handey: A robot system that recognizes, plans, and manipulates , 1987, Proceedings. 1987 IEEE International Conference on Robotics and Automation.

[22]  Oussama Khatib,et al.  A unified approach for motion and force control of robot manipulators: The operational space formulation , 1987, IEEE J. Robotics Autom..

[23]  S. Sastry,et al.  Task oriented optimal grasping by multifingered robot hands , 1987, Proceedings. 1987 IEEE International Conference on Robotics and Automation.

[24]  Joel W. Burdick,et al.  On the inverse kinematics of redundant manipulators: characterization of the self-motion manifolds , 1989, Proceedings, 1989 International Conference on Robotics and Automation.

[25]  Van-Duc Nguyen,et al.  Constructing Stable Grasps , 1989, Int. J. Robotics Res..

[26]  John F. Canny,et al.  On computability of fine motion plans , 1989, Proceedings, 1989 International Conference on Robotics and Automation.

[27]  Christopher G. Atkeson,et al.  Task-level robot learning: juggling a tennis ball more accurately , 1989, Proceedings, 1989 International Conference on Robotics and Automation.

[28]  Sharon A. Stansfield,et al.  Robotic Grasping of Unknown Objects: A Knowledge-based Approach , 1991, Int. J. Robotics Res..

[29]  David J. Montana,et al.  Contact stability for two-fingered grasps , 1992, IEEE Trans. Robotics Autom..

[30]  P. L. Combettes The foundations of set theoretic estimation , 1993 .

[31]  John T. Wen,et al.  A global approach to path planning for redundant manipulators , 1993, [1993] Proceedings IEEE International Conference on Robotics and Automation.

[32]  Jean-Claude Latombe,et al.  Planning motions with intentions , 1994, SIGGRAPH.

[33]  Min Zhao,et al.  Mobile manipulator path planning by a genetic algorithm , 1994, J. Field Robotics.

[34]  Rachid Alami,et al.  Two manipulation planning algorithms , 1995 .

[35]  Bruno Siciliano,et al.  Modeling and Control of Robot Manipulators , 1995 .

[36]  Lydia E. Kavraki,et al.  Randomized query processing in robot path planning , 1995, STOC '95.

[37]  B. Faverjon,et al.  Probabilistic Roadmaps for Path Planning in High-Dimensional Con(cid:12)guration Spaces , 1996 .

[38]  P. Svestka On probabilistic completeness and expected complexity for probabilistic path planning , 1996 .

[39]  Ali M. S. Zalzala,et al.  A genetic approach to motion planning of redundant mobile manipulator systems considering safety and configuration , 1997, J. Field Robotics.

[40]  Robert B. Fisher,et al.  Estimating 3-D rigid body transformations: a comparison of four major algorithms , 1997, Machine Vision and Applications.

[41]  Jean-Daniel Boissonnat,et al.  On Computing Four-Finger Equilibrium and Force-Closure Grasps of Polyhedral Objects , 1997, Int. J. Robotics Res..

[42]  Dorothea Heiss-Czedik,et al.  An Introduction to Genetic Algorithms. , 1997, Artificial Life.

[43]  Stefan Schaal,et al.  Learning tasks from a single demonstration , 1997, Proceedings of International Conference on Robotics and Automation.

[44]  Yong K. Hwang,et al.  SANDROS: a dynamic graph search algorithm for motion planning , 1998, IEEE Trans. Robotics Autom..

[45]  Dinesh Manocha,et al.  Fast Proximity Queries with Swept Sphere Volumes , 1999 .

[46]  David Hsu,et al.  Placing a robot manipulator amid obstacles for optimized execution , 1999, Proceedings of the 1999 IEEE International Symposium on Assembly and Task Planning (ISATP'99) (Cat. No.99TH8470).

[47]  Steven M. LaValle,et al.  Rapidly-Exploring Random Trees: Progress and Prospects , 2000 .

[48]  Steven M. LaValle,et al.  RRT-connect: An efficient approach to single-query path planning , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[49]  Jean-Claude Latombe,et al.  Capturing molecular energy landscapes with probabilistic conformational roadmaps , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[50]  Matthew T. Mason,et al.  Mechanics of Robotic Manipulation , 2001 .

[51]  Lydia E. Kavraki,et al.  Randomized path planning for linkages with closed kinematic chains , 2001, IEEE Trans. Robotics Autom..

[52]  Yoshihiko Nakamura,et al.  Whole-body cooperative balancing of humanoid robot using COG Jacobian , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[53]  Marilena Vendittelli,et al.  Probabilistic motion planning for redundant robots along given end-effector paths , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[54]  Michael A. Arbib,et al.  Schema design and implementation of the grasp-related mirror neuron system , 2002, Biological Cybernetics.

[55]  David Hsu,et al.  The bridge test for sampling narrow passages with probabilistic roadmap planners , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[56]  Henrik I. Christensen,et al.  Automatic grasp planning using shape primitives , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[57]  Jun Wang,et al.  Synthesis of force-closure grasps on 3-D objects based on the Q distance , 2003, IEEE Trans. Robotics Autom..

[58]  Reid G. Simmons,et al.  Approaches for heuristically biasing RRT growth , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[59]  Mikko Sallinen,et al.  Modelling and estimation of spatial relationships in sensor-based robot workcells , 2003 .

[60]  Katsu Yamane,et al.  Synthesizing animations of human manipulation tasks , 2004, ACM Trans. Graph..

[61]  Gordon Cheng,et al.  Learning tasks from observation and practice , 2004, Robotics Auton. Syst..

[62]  A. Neumaier Complete search in continuous global optimization and constraint satisfaction , 2004, Acta Numerica.

[63]  Nancy S. Pollard,et al.  Closure and Quality Equivalence for Efficient Synthesis of Grasps from Examples , 2004, Int. J. Robotics Res..

[64]  Thierry Siméon,et al.  Sampling-Based Motion Planning under Kinematic Loop-Closure Constraints , 2004, WAFR.

[65]  Thierry Siméon,et al.  Manipulation Planning with Probabilistic Roadmaps , 2004, Int. J. Robotics Res..

[66]  Peter K. Allen,et al.  An SVM learning approach to robotic grasping , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[67]  Masayuki Inaba,et al.  Environment manipulation planner for humanoid robots using task graph that generates action sequence , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[68]  Rachid Alami,et al.  A Robot Task Planner that Merges Symbolic and Geometric Reasoning , 2004, ECAI.

[69]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[70]  Kamal K. Gupta,et al.  Path planning with general end-effector constraints: using task space to guide configuration space search , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[71]  Giuseppe Oriolo,et al.  Motion Planning for Mobile Manipulators along Given End-effector Paths , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[72]  Oussama Khatib,et al.  Synthesis of Whole-Body Behaviors through Hierarchical Control of Behavioral Primitives , 2005, Int. J. Humanoid Robotics.

[73]  Yutaka Hirano,et al.  Image-based object recognition and dexterous hand/arm motion planning using RRTs for grasping in cluttered scene , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[74]  Anthony Stentz,et al.  Anytime RRTs , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[75]  Hannes Bleuler,et al.  Randomised Rough-Terrain Robot Motion Planning , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[76]  Nancy M. Amato,et al.  An obstacle-based rapidly-exploring random tree , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[77]  Tamim Asfour,et al.  Integrated Grasp Planning and Visual Object Localization For a Humanoid Robot with Five-Fingered Hands , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[78]  Tomás Lozano-Pérez,et al.  Imitation Learning of Whole-Body Grasps , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[79]  Steven M. LaValle,et al.  Planning algorithms , 2006 .

[80]  Ashutosh Saxena,et al.  Robotic Grasping of Novel Objects , 2006, NIPS.

[81]  Ho-Lun Cheng,et al.  Multi-level free-space dilation for sampling narrow passages in PRM planning , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[82]  Victor Ng-Thow-Hing,et al.  Toward Interactive Reaching in Static Environments for Humanoid Robots , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[83]  Tamim Asfour,et al.  An integrated approach to inverse kinematics and path planning for redundant manipulators , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[84]  Wen-Han Qian,et al.  An Enhanced Ray-Shooting Approach to Force-Closure Problems , 2006 .

[85]  Joris De Schutter,et al.  Constraint-based Task Specification and Estimation for Sensor-Based Robot Systems in the Presence of Geometric Uncertainty , 2007, Int. J. Robotics Res..

[86]  Stefan Edelkamp,et al.  Automated Planning: Theory and Practice , 2007, Künstliche Intell..

[87]  Dmitry Berenson,et al.  Grasp planning in complex scenes , 2007, 2007 7th IEEE-RAS International Conference on Humanoid Robots.

[88]  Aude Billard,et al.  On Learning, Representing, and Generalizing a Task in a Humanoid Robot , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[89]  Siddhartha S. Srinivasa,et al.  Randomized path planning for redundant manipulators without inverse kinematics , 2007, 2007 7th IEEE-RAS International Conference on Humanoid Robots.

[90]  Rachid Alami,et al.  A Human Aware Mobile Robot Motion Planner , 2007, IEEE Transactions on Robotics.

[91]  Ying Li,et al.  Data-Driven Grasp Synthesis Using Shape Matching and Task-Based Pruning , 2007, IEEE Transactions on Visualization and Computer Graphics.

[92]  Siddhartha S. Srinivasa,et al.  Imitation learning for locomotion and manipulation , 2007, 2007 7th IEEE-RAS International Conference on Humanoid Robots.

[93]  Mark H. Overmars,et al.  Creating High-quality Paths for Motion Planning , 2007, Int. J. Robotics Res..

[94]  Mike Stilman,et al.  Task constrained motion planning in robot joint space , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[95]  Nikolaos G. Tsagarakis,et al.  iCub: the design and realization of an open humanoid platform for cognitive and neuroscience research , 2007, Adv. Robotics.

[96]  Tamim Asfour,et al.  Manipulation Planning Among Movable Obstacles , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[97]  Vijay Kumar,et al.  Cooperative Towing with Multiple Robots , 2008, WAFR.

[98]  Dmitry Berenson,et al.  An optimization approach to planning for mobile manipulation , 2008, 2008 IEEE International Conference on Robotics and Automation.

[99]  Jing Xiao,et al.  Real-Time Adaptive Motion Planning (RAMP) of Mobile Manipulators in Dynamic Environments With Unforeseen Changes , 2008, IEEE Transactions on Robotics.

[100]  Paul Umbanhowar,et al.  Friction-Induced Velocity Fields for Point Parts Sliding on a Rigid Oscillated Plate , 2009, Int. J. Robotics Res..

[101]  Siddhartha S. Srinivasa,et al.  Manipulation planning with caging grasps , 2008, Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots.

[102]  Sethu Vijayakumar,et al.  Learning potential-based policies from constrained motion , 2008, Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots.

[103]  Jean-Paul Laumond,et al.  Control of Probabilistic Diffusion in Motion Planning , 2008, WAFR.

[104]  Tamim Asfour,et al.  Toward humanoid manipulation in human-centred environments , 2008, Robotics Auton. Syst..

[105]  Oliver Brock,et al.  Learning to Manipulate Articulated Objects in Unstructured Environments Using a Grounded Relational Representation , 2008, Robotics: Science and Systems.

[106]  Siddhartha S. Srinivasa,et al.  Grasp synthesis in cluttered environments for dexterous hands , 2008, Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots.

[107]  Wolfram Burgard,et al.  Learning Kinematic Models for Articulated Objects , 2009, IJCAI.

[108]  Siddhartha S. Srinivasa,et al.  CHOMP: Gradient optimization techniques for efficient motion planning , 2009, 2009 IEEE International Conference on Robotics and Automation.

[109]  Thomas H. Vose,et al.  Friction-Induced Velocity Fields for Point Parts Sliding on a Rigid Oscillated Plate , 2009, Int. J. Robotics Res..

[110]  Siddhartha S. Srinivasa,et al.  Pose-constrained whole-body planning using Task Space Region Chains , 2009, 2009 9th IEEE-RAS International Conference on Humanoid Robots.

[111]  Siddhartha S. Srinivasa,et al.  Manipulation planning with Workspace Goal Regions , 2009, 2009 IEEE International Conference on Robotics and Automation.

[112]  Abderrahmane Kheddar,et al.  Contact planning for acyclic motion with tasks constraints , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[113]  Matei T. Ciocarlie,et al.  The Columbia grasp database , 2009, 2009 IEEE International Conference on Robotics and Automation.

[114]  Siddhartha S. Srinivasa,et al.  Manipulation planning on constraint manifolds , 2009, 2009 IEEE International Conference on Robotics and Automation.

[115]  Florent Lamiraux,et al.  Whole-body task planning for a humanoid robot: a way to integrate collision avoidance , 2009, 2009 9th IEEE-RAS International Conference on Humanoid Robots.

[116]  Siddhartha S. Srinivasa,et al.  Addressing pose uncertainty in manipulation planning using Task Space Regions , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[117]  Thierry Siméon,et al.  Sampling-Based Path Planning on Configuration-Space Costmaps , 2010, IEEE Transactions on Robotics.

[118]  Siddhartha S. Srinivasa,et al.  Probabilistically complete planning with end-effector pose constraints , 2010, 2010 IEEE International Conference on Robotics and Automation.

[119]  Takeo Kanade,et al.  Automated Construction of Robotic Manipulation Programs , 2010 .

[120]  Dinesh Manocha,et al.  Retraction-based RRT planner for articulated models , 2010, 2010 IEEE International Conference on Robotics and Automation.

[121]  J. Trinkle,et al.  dvc 3 D : a three dimensional physical simulation tool for rigid bodies with contacts and Coulomb friction , 2010 .

[122]  S. Srinivasa,et al.  Push-grasping with dexterous hands: Mechanics and a method , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[123]  Jean-Claude Latombe,et al.  Multi-modal Motion Planning in Non-expansive Spaces , 2010, Int. J. Robotics Res..

[124]  Subramanian Ramamoorthy,et al.  Constrained geodesic trajectory generation on learnt skill manifolds , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[125]  Siddhartha S. Srinivasa,et al.  Addressing cost-space chasms in manipulation planning , 2011, 2011 IEEE International Conference on Robotics and Automation.

[126]  Dylan A. Shell,et al.  An evaluation of methods for modeling contact in multibody simulation , 2011, 2011 IEEE International Conference on Robotics and Automation.

[127]  Dima Damen,et al.  Egocentric Real-time Workspace Monitoring using an RGB-D camera , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[128]  Siddhartha S. Srinivasa,et al.  Constellation - An algorithm for finding robot configurations that satisfy multiple constraints , 2012, 2012 IEEE International Conference on Robotics and Automation.