Prioritized Multi-Objective Robot Control

Humans are capable of pursuing multiple objectives simultaneously in everyday life, thereby flexibly prioritizing motion tasks and constraints among others. A major goal of current robotics research is to equip highly redundant robots with similar adaptive behavior. The aim of this thesis is to endow such robots with the abilities to optimize priorities for possibly conflicting objectives, to smoothly rearrange such priorities and to decouple the objectives related to motion generation from the objective of simultaneous interaction with the environment through forces. A commonly adopted approach is to tune priorities by hand once which then remain constant over time. In contrast to such constant prioritization, this thesis promotes a different view: A framework for automated learning soft priorities is introduced. Employing state-of-the-art optimization, different sets of priorities are evaluated offline and improved until the prioritized superposition of all underlying controllers satisfies a desired high-level goal. As an alternative approach to soft prioritization with scalar weights, the well-known Stack-of-Tasks prioritization scheme relies on projectors and enforces strict priorities. The second part of this thesis performs a thorough formal analysis of projectors and proposes to smoothly shape between idempotent matrix operators. A novel prioritization scheme is presented with the help of this method, which generalizes previous approaches implementing either strict or soft priorities. It enables to insert new or remove outdated objectives if necessary and allows to rearrange priorities continuously online without inertia coupling while offering the ability to control the interference between objectives. Finally, underactuated robots subject to contact constraints are studied. A projection operator is derived which enables to decouple contact wrench control from constraint-consistent motion generation for robots with passive or virtual joints. This is essential for controlling legged robots which are represented by a floating-base. Moreover, this thesis shows that the same principle holds for underactuated grasping scenarios. The approach is validated with a large panel of experiments on various robot platforms both in simulation and the real-world. This thesis provides numerous tools that endows existing and future robot systems with ever-increasing degrees of freedom to create complex motion aiming to interact with their environment effectively.

[1]  Stefan Schaal,et al.  Optimal distribution of contact forces with inverse-dynamics control , 2013, Int. J. Robotics Res..

[2]  Oussama Khatib,et al.  Control of Redundant Robots Under Hard Joint Constraints: Saturation in the Null Space , 2015, IEEE Transactions on Robotics.

[3]  Vincent Padois,et al.  Generalized hierarchical control , 2015, Autonomous Robots.

[4]  Daniel E. Koditschek,et al.  Legged Self-Manipulation , 2013, IEEE Access.

[5]  Alessandro De Luca,et al.  Unilateral constraints in the Reverse Priority redundancy resolution method , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[6]  Farhad Aghili,et al.  A unified approach for inverse and direct dynamics of constrained multibody systems based on linear projection operator: applications to control and simulation , 2005, IEEE Transactions on Robotics.

[7]  Masaru Uchiyama,et al.  Symmetric kinematic formulation and non-master/slave coordinated control of two-arm robots , 1992, Adv. Robotics.

[8]  Chun-Yi Su,et al.  Control of constrained robots subject to unilateral contacts and friction cone constraints , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[9]  Alexander Dietrich,et al.  An overview of null space projections for redundant, torque-controlled robots , 2015, Int. J. Robotics Res..

[10]  Jochen J. Steil,et al.  Optimization of Manipulator Base Placements for Multi-Arm Grasping , 2018 .

[11]  Darwin G. Caldwell,et al.  Learning Competing Constraints and Task Priorities from Demonstrations of Bimanual Skills , 2017, ArXiv.

[12]  Olivier Sigaud,et al.  Multiple task optimization using dynamical movement primitives for whole-body reactive control , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[13]  Valerio Ortenzi,et al.  An experimental study of robot control during environmental contacts based on projected operational space dynamics , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[14]  Olivier Sigaud,et al.  Variance modulated task prioritization in Whole-Body Control , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[15]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[16]  Jochen J. Steil,et al.  Multiple task optimization with a mixture of controllers for motion generation , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[17]  Antal K. Bejczy,et al.  Efficient Jacobian inversion for the control of simple robot manipulators , 1988, Proceedings. 1988 IEEE International Conference on Robotics and Automation.

[18]  Giuseppe Oriolo,et al.  Learning soft task priorities for control of redundant robots , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[19]  S. S. Kim,et al.  QR Decomposition for State Space Representation of Constrained Mechanical Dynamic Systems , 1986 .

[20]  P. Chiacchio,et al.  Six-DOF Impedance Control of Dual-Arm Cooperative Manipulators , 2008, IEEE/ASME Transactions on Mechatronics.

[21]  Giuseppe Muscio,et al.  Distributed cooperative object parameter estimation and manipulation without explicit communication , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[22]  Jun Nakanishi,et al.  Inverse kinematics with floating base and constraints for full body humanoid robot control , 2008, Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots.

[23]  Nicolas Mansard,et al.  Intermediate Desired Value Approach for Task Transition of Robots in Kinematic Control , 2012, IEEE Transactions on Robotics.

[24]  Darwin G. Caldwell,et al.  Probabilistic Learning of Torque Controllers from Kinematic and Force Constraints , 2017, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[25]  Nikolaos G. Tsagarakis,et al.  Development of a dynamic simulator for a compliant humanoid robot based on a symbolic multibody approach , 2013, 2013 IEEE International Conference on Mechatronics (ICM).

[26]  Gerd Hirzinger,et al.  Posture and balance control for biped robots based on contact force optimization , 2011, 2011 11th IEEE-RAS International Conference on Humanoid Robots.

[27]  Dana Kulic,et al.  Incremental learning of full body motion primitives and their sequencing through human motion observation , 2012, Int. J. Robotics Res..

[28]  Olivier Sigaud,et al.  Robot Skill Learning: From Reinforcement Learning to Evolution Strategies , 2013, Paladyn J. Behav. Robotics.

[29]  Aude Billard,et al.  Learning motion dynamics to catch a moving object , 2010, 2010 10th IEEE-RAS International Conference on Humanoid Robots.

[30]  Oussama Khatib,et al.  A whole-body control framework for humanoids operating in human environments , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[31]  Nicolas Mansard,et al.  Task Sequencing for High-Level Sensor-Based Control , 2007, IEEE Transactions on Robotics.

[32]  Yoshihiko Nakamura,et al.  Inverse kinematic solutions with singularity robustness for robot manipulator control , 1986 .

[33]  Vincent Padois,et al.  Synthesis of complex humanoid whole-body behavior: A focus on sequencing and tasks transitions , 2011, 2011 IEEE International Conference on Robotics and Automation.

[34]  Darwin G. Caldwell,et al.  Learning bimanual end-effector poses from demonstrations using task-parameterized dynamical systems , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[35]  Ian D. Walker,et al.  Overview of damped least-squares methods for inverse kinematics of robot manipulators , 1995, J. Intell. Robotic Syst..

[36]  Nikos G. Tsagarakis,et al.  An attractor-based Whole-Body Motion Control (WBMC) system for humanoid robots , 2013, 2013 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids).

[37]  Bernd Henze,et al.  Passivity-based whole-body balancing for torque-controlled humanoid robots in multi-contact scenarios , 2016, Int. J. Robotics Res..

[38]  Tadej Petric,et al.  Smooth continuous transition between tasks on a kinematic control level: Obstacle avoidance as a control problem , 2013, Robotics Auton. Syst..

[39]  Jaeheung Park,et al.  Robot Control near Singularity and Joint Limit Using a Continuous Task Transition Algorithm , 2013 .

[40]  Jochen J. Steil,et al.  Continuously Shaping Projections and Operational Space Tasks , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[41]  Giuseppe Casalino,et al.  A Novel Practical Technique to Integrate Inequality Control Objectives and Task Transitions in Priority Based Control , 2016, J. Intell. Robotic Syst..

[42]  Pierre-Brice Wieber,et al.  Hierarchical quadratic programming: Fast online humanoid-robot motion generation , 2014, Int. J. Robotics Res..

[43]  Alexander Dietrich,et al.  On continuous null space projections for torque-based, hierarchical, multi-objective manipulation , 2012, 2012 IEEE International Conference on Robotics and Automation.

[44]  Darwin G. Caldwell,et al.  Learning Task Priorities from Demonstrations , 2017, IEEE Transactions on Robotics.

[45]  Jaemin Lee,et al.  A continuous task transition algorithm for operational space control framework , 2012, 2012 9th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI).

[46]  Christopher G. Atkeson,et al.  Estimation of Inertial Parameters of Manipulator Loads and Links , 1986 .

[47]  Nicolas Mansard,et al.  Continuity of Varying-Feature-Set Control Laws , 2009, IEEE Transactions on Automatic Control.

[48]  Sandra Hirche,et al.  Internal Force Analysis and Load Distribution for Cooperative Multi-Robot Manipulation , 2015, IEEE Transactions on Robotics.

[49]  Jean-Jacques E. Slotine,et al.  A general framework for managing multiple tasks in highly redundant robotic systems , 1991, Fifth International Conference on Advanced Robotics 'Robots in Unstructured Environments.

[50]  Gustavo Arechavaleta,et al.  Time parametrization of prioritized inverse kinematics based on terminal attractors , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[51]  Sebastian Wrede,et al.  Domain-Specific Language Modularization Scheme Applied to a Multi-Arm Robotics Use-Case , 2017 .

[52]  Alexander Dietrich,et al.  Prioritized multi-task compliance control of redundant manipulators , 2015, Autom..

[53]  Richard M. Murray,et al.  A Mathematical Introduction to Robotic Manipulation , 1994 .

[54]  Oussama Khatib,et al.  Compliant Control of Multicontact and Center-of-Mass Behaviors in Humanoid Robots , 2010, IEEE Transactions on Robotics.

[55]  Oussama Khatib,et al.  A Unified Approach to Integrate Unilateral Constraints in the Stack of Tasks , 2009, IEEE Transactions on Robotics.

[56]  Nicolas Mansard,et al.  Regularized Hierarchical Differential Dynamic Programming , 2017, IEEE Transactions on Robotics.

[57]  Stefan Schaal,et al.  Inverse dynamics control of floating-base robots with external constraints: A unified view , 2011, 2011 IEEE International Conference on Robotics and Automation.

[58]  Jochen J. Steil,et al.  Continuous task-priority rearrangement during motion execution with a mixture of torque controllers , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[59]  Jochen J. Steil,et al.  A Comparison of Null-space Projection and Mixture of Torque Controllers for Motion Generation , 2016 .

[60]  Abderrahmane Kheddar,et al.  Multi-robot and task-space force control with quadratic programming , 2017 .

[61]  Vincent Padois,et al.  Generalized projector for task priority transitions during hierarchical control , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[62]  Daniel Sebastian Leidner,et al.  Cognitive Reasoning for Compliant Robot Manipulation , 2018, Springer Tracts in Advanced Robotics.

[63]  Jan Peters,et al.  Learning table tennis with a Mixture of Motor Primitives , 2010, 2010 10th IEEE-RAS International Conference on Humanoid Robots.

[64]  Gerd Hirzinger,et al.  Analysis and experimental evaluation of the Intrinsically Passive Controller (IPC) for multifingered hands , 2008, 2008 IEEE International Conference on Robotics and Automation.

[65]  Alin Albu-Schäffer,et al.  Cartesian impedance control of redundant robots: recent results with the DLR-light-weight-arms , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[66]  Jun Nakanishi,et al.  Learning Attractor Landscapes for Learning Motor Primitives , 2002, NIPS.

[67]  Paul Fisette,et al.  ROBOTRAN: a powerful symbolic gnerator of multibody models , 2013 .

[68]  Carl D. Meyer,et al.  Matrix Analysis and Applied Linear Algebra , 2000 .

[69]  Ludovic Righetti,et al.  Operational Space Control of Constrained and Underactuated Systems , 2011, Robotics: Science and Systems.

[70]  Abderrahmane Kheddar,et al.  Using a multi-objective controller to synthesize simulated humanoid robot motion with changing contact configurations , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[71]  Neville Hogan,et al.  Impedance control - An approach to manipulation. I - Theory. II - Implementation. III - Applications , 1985 .

[72]  Giuseppe Oriolo,et al.  Learning soft task priorities for safe control of humanoid robots with constrained stochastic optimization , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[73]  Alexander Herzog,et al.  Momentum control with hierarchical inverse dynamics on a torque-controlled humanoid , 2014, Autonomous Robots.

[74]  Oussama Khatib,et al.  Synthesis of Whole-Body Behaviors through Hierarchical Control of Behavioral Primitives , 2005, Int. J. Humanoid Robotics.

[75]  Alexander Dietrich,et al.  Reactive Whole-Body Control: Dynamic Mobile Manipulation Using a Large Number of Actuated Degrees of Freedom , 2012, IEEE Robotics & Automation Magazine.

[76]  Andrej Gams,et al.  Bimanual human robot cooperation with adaptive stiffness control , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[77]  Jochen J. Steil,et al.  Efficient policy search with a parameterized skill memory , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[78]  Donald Goldfarb,et al.  A numerically stable dual method for solving strictly convex quadratic programs , 1983, Math. Program..

[79]  Hsiu-Chin Lin,et al.  A Projected Inverse Dynamics Approach for Multi-Arm Cartesian Impedance Control , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[80]  Jun Nakanishi,et al.  Operational Space Control: A Theoretical and Empirical Comparison , 2008, Int. J. Robotics Res..

[81]  Peter Stone,et al.  Dynamic behaviors on the NAO robot with closed-loop whole body operational space control , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[82]  Sylvain Calinon,et al.  Learning Control , 2018, Encyclopedia of Machine Learning and Data Mining.

[83]  Oussama Khatib,et al.  A unified approach for motion and force control of robot manipulators: The operational space formulation , 1987, IEEE J. Robotics Autom..

[84]  Eren Erdal Aksoy,et al.  On the Dualities Between Grasping and Whole-Body Loco-Manipulation Tasks , 2015, ISRR.

[85]  Gustavo Arechavaleta,et al.  Continuous kinematic control with terminal attractors for handling task transitions of redundant robots , 2013, 2013 IEEE International Conference on Robotics and Automation.

[86]  Helge J. Ritter,et al.  Task-oriented quality measures for dextrous grasping , 2005, 2005 International Symposium on Computational Intelligence in Robotics and Automation.

[87]  Tsuneo Yoshikawa Virtual truss model for characterization of internal forces for multiple finger grasps , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[88]  Abderrahmane Kheddar,et al.  Interlinked Visual Tracking and Robotic Manipulation of Articulated Objects , 2018, IEEE Robotics and Automation Letters.

[89]  Eiichi Yoshida,et al.  Real-time smooth task transitions for hierarchical inverse kinematics , 2013, 2013 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids).

[90]  Pierre-Brice Wieber,et al.  Analysis of the discontinuities in prioritized tasks-space control under discreet task scheduling operations , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[91]  Thomas Mergner,et al.  A neurological view on reactive human stance control , 2010, Annu. Rev. Control..

[92]  Jun Nakanishi,et al.  Dynamical Movement Primitives: Learning Attractor Models for Motor Behaviors , 2013, Neural Computation.

[93]  Vincent Padois,et al.  Whole-body hierarchical motion and force control for humanoid robots , 2016, Auton. Robots.

[94]  Dongheui Lee,et al.  Prioritized Inverse Kinematics with Multiple Task Definitions , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[95]  Oussama Khatib,et al.  Motion control of redundant robots under joint constraints: Saturation in the Null Space , 2012, 2012 IEEE International Conference on Robotics and Automation.

[96]  Abderrahmane Kheddar,et al.  Multi-Character Physical and Behavioral Interactions Controller , 2017, IEEE Transactions on Visualization and Computer Graphics.

[97]  Abderrahmane Kheddar,et al.  On Weight-Prioritized Multitask Control of Humanoid Robots , 2018, IEEE Transactions on Automatic Control.

[98]  Allison M. Okamura,et al.  An overview of dexterous manipulation , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[99]  Pierre-Brice Wieber,et al.  Kinematic Control of Redundant Manipulators: Generalizing the Task-Priority Framework to Inequality Task , 2011, IEEE Transactions on Robotics.

[100]  Majid Nili Ahmadabadi,et al.  A dynamic object manipulation approach to dynamic biped locomotion , 2008, Robotics Auton. Syst..

[101]  Peter Fankhauser,et al.  ANYmal - a highly mobile and dynamic quadrupedal robot , 2016, IROS 2016.

[102]  Darwin G. Caldwell,et al.  Robot motor skill coordination with EM-based Reinforcement Learning , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[103]  Nikolaos G. Tsagarakis,et al.  Multi-Priority Cartesian Impedance Control Based on Quadratic Programming Optimization , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[104]  Jochen J. Steil,et al.  Modeling and Control of Multi-Arm and Multi-Leg Robots: Compensating for Object Dynamics During Grasping , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[105]  Chee-Meng Chew,et al.  Virtual Model Control: An Intuitive Approach for Bipedal Locomotion , 2001, Int. J. Robotics Res..

[106]  Darwin G. Caldwell,et al.  High-slope terrain locomotion for torque-controlled quadruped robots , 2016, Autonomous Robots.

[107]  Jochen J. Steil,et al.  Self-supervised bootstrapping of a movement primitive library from complex trajectories , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[108]  Oliver Brock,et al.  Task-consistent obstacle avoidance and motion behavior for mobile manipulation , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[109]  Soonwook Hwang,et al.  Balancing of humanoid robot using contact force/moment control by task-oriented whole body control framework , 2016, Auton. Robots.

[110]  Mario Castelán,et al.  Hierarchical Task-Based Control of Multirobot Systems With Terminal Attractors , 2017, IEEE Transactions on Control Systems Technology.

[111]  Hsiu-Chin Lin,et al.  A Model-Based Hierarchical Controller for Legged Systems Subject to External Disturbances , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[112]  Alexander Dietrich,et al.  The Hierarchical Operational Space Formulation: Stability Analysis for the Regulation Case , 2018, IEEE Robotics and Automation Letters.