Towards Minimal Intervention Control with Competing Constraints

As many imitation learning algorithms focus on pure trajectory generation in either Cartesian space or joint space, the problem of considering competing trajectory constraints from both spaces still presents several challenges. In particular, when perturbations are applied to the robot, the underlying controller should take into account the importance of each space for the task execution, and compute the control effort accordingly. However, no such controller formulation exists. In this paper, we provide a minimal intervention control strategy that simultaneously addresses the problems of optimal control and competing constraints between Cartesian and joint spaces. In light of the inconsistency between Cartesian and joint constraints, we exploit the robot null space from an informationtheory perspective so as to reduce the corresponding conflict. An optimal solution to the aforementioned controller is derived and furthermore a connection to the classical finite horizon linear quadratic regulator (LQR) is provided. Finally, a writing task in a simulated robot verifies the effectiveness of our approach.

[1]  Jun Nakanishi,et al.  Dynamical Movement Primitives: Learning Attractor Models for Motor Behaviors , 2013, Neural Computation.

[2]  Yasemin Altun,et al.  Relative Entropy Policy Search , 2010 .

[3]  Stefan Schaal,et al.  Learning and generalization of motor skills by learning from demonstration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[4]  Sandra Hirche,et al.  Risk-Sensitive Optimal Feedback Control for Haptic Assistance , 2012, 2012 IEEE International Conference on Robotics and Automation.

[5]  Bernhard Schölkopf,et al.  Learning optimal striking points for a ping-pong playing robot , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[6]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[7]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[8]  Peters Jan,et al.  Jointly learning trajectory generation and hitting point prediction in robot table tennis , 2016 .

[9]  Jochen J. Steil,et al.  Task-level imitation learning using variance-based movement optimization , 2009, 2009 IEEE International Conference on Robotics and Automation.

[10]  Darwin G. Caldwell,et al.  A task-parameterized probabilistic model with minimal intervention control , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[11]  Darwin G. Caldwell,et al.  Hybrid Probabilistic Trajectory Optimization Using Null-Space Exploration , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[12]  Darwin G. Caldwell,et al.  Learning Task Priorities from Demonstrations , 2017, IEEE Transactions on Robotics.

[13]  Aude Billard,et al.  Statistical Learning by Imitation of Competing Constraints in Joint Space and Task Space , 2009, Adv. Robotics.

[14]  John M. Hollerbach,et al.  Redundancy resolution of manipulators through torque optimization , 1987, IEEE J. Robotics Autom..

[15]  Stefan Schaal,et al.  A Generalized Path Integral Control Approach to Reinforcement Learning , 2010, J. Mach. Learn. Res..

[16]  Olivier Sigaud,et al.  Robot Skill Learning: From Reinforcement Learning to Evolution Strategies , 2013, Paladyn J. Behav. Robotics.

[17]  Darwin G. Caldwell,et al.  An Uncertainty-Aware Minimal Intervention Control Strategy Learned from Demonstrations , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[18]  Stefan Schaal,et al.  Policy Gradient Methods for Robotics , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[19]  Darwin G. Caldwell,et al.  Kernelized movement primitives , 2017, Int. J. Robotics Res..

[20]  Tamim Asfour,et al.  Task-oriented generalization of dynamic movement primitive , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[21]  Sylvain Calinon,et al.  A tutorial on task-parameterized movement learning and retrieval , 2016, Intell. Serv. Robotics.

[22]  Sergey Levine,et al.  Learning Neural Network Policies with Guided Policy Search under Unknown Dynamics , 2014, NIPS.

[23]  Jan Peters,et al.  Probabilistic Movement Primitives , 2013, NIPS.