Control and Learning of Compliant Manipulation Skills

Humans demonstrate an impressive capability to manipulate fragile objects without damaging them, graciously controlling the force and position of hands or tools. Traditionally, robotics has favored position control over force control to produce fast, accurate and repeatable motion. For extending the applicability of robotic manipulators outside the strictly controlled environments of industrial work cells, position control is inadequate. Tasks that involve contact with objects whose positions are not known with perfect certainty require a controller that regulates the relationship between positional deviations and forces on the robot. This problem is formalized in the impedance control framework, which focuses the robot control problem on the interaction between the robot and its environment. By adjusting the impedance parameters, the behavior of the robot can be adapted to the need of the task. However, it is often difficult to specify formally how the impedance should vary for best performance. Furthermore, fast it can be shown that careless variation of the impedance can lead to unstable regulation or tracking even in free motion. In the first part of the thesis, the problem of how to define a varying impedance for a task is addressed. A haptic human-robot interface that allows a human supervisor to teach impedance variations by physically interacting with the robot during task execution is introduced. It is shown that the interface can be used to enhance the performance in several manipulation tasks. Then, the problem of stable control with varying impedance is addressed. Along with a theoretical discussion on this topic, a sufficient condition for stable varying stiffness and damping is provided. In the second part of the thesis, we explore more complex manipulation scenarios via online generation of the robot trajectory. This is done along two axes 1) learning how to react to contact forces in insertion tasks which are crucial for assembly operations and 2) autonomous Dynamical Systems (DS) for motion representation with the capability to encode a family of trajectories rather than a fixed, time-dependent reference. A novel framework for task representation using DS is introduced, termed Locally Modulated Dynamical Systems (LMDS). LMDS differs from existing DS estimation algorithms in that it supports non-parametric and incremental learning all the while guaranteeing that the resulting DS is globally stable at an attractor point. To combine the advantages of DS motion generation with impedance control, a novel controller for tasks described by first order DS is proposed. The controller is passive, and has the properties of an impedance controller with the added flexibility of a DS motion representation instead of a time-indexed trajectory.

[1]  Hassan K. Khalil,et al.  Nonlinear Systems Third Edition , 2008 .

[2]  I. Cathers,et al.  Tracking performance with sinusoidal and irregular targets under different conditions of peripheral feedback , 1996, Experimental Brain Research.

[3]  J. Salisbury,et al.  Active stiffness control of a manipulator in cartesian coordinates , 1980, 1980 19th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[4]  H. Gomi,et al.  Multijoint muscle regulation mechanisms examined by measured human arm stiffness and EMG signals. , 1999, Journal of neurophysiology.

[5]  Daniel E. Whitney,et al.  Quasi-Static Assembly of Compliantly Supported Rigid Parts , 1982 .

[6]  O. Khatib,et al.  Real-Time Obstacle Avoidance for Manipulators and Mobile Robots , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[7]  Darwin G. Caldwell,et al.  Compliant skills acquisition and multi-optima policy search with EM-based reinforcement learning , 2013, Robotics Auton. Syst..

[8]  Joseph Duffy,et al.  Hybrid Twist and Wrench Control for a Robotic Manipulator , 1988 .

[9]  Aude Billard,et al.  On learning the statistical representation of a task and generalizing it to various contexts , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[10]  Darwin G. Caldwell,et al.  A task-parameterized probabilistic model with minimal intervention control , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[11]  Marnix Nuttin,et al.  Learning Complex Tasks Using a Stepwise Approach , 1999, J. Intell. Robotic Syst..

[12]  Aude Billard,et al.  Coupled dynamical system based arm-hand grasping model for learning fast adaptation strategies , 2012, Robotics Auton. Syst..

[13]  Dana Kulic,et al.  Incremental Learning, Clustering and Hierarchy Formation of Whole Body Motion Patterns using Adaptive Hidden Markov Chains , 2008, Int. J. Robotics Res..

[14]  John J. Craig,et al.  Hybrid position/force control of manipulators , 1981 .

[15]  Aude Billard,et al.  Online learning of varying stiffness through physical human-robot interaction , 2012, 2012 IEEE International Conference on Robotics and Automation.

[16]  Aude Billard,et al.  Incremental learning of gestures by imitation in a humanoid robot , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[17]  Geoffrey E. Hinton,et al.  A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants , 1998, Learning in Graphical Models.

[18]  P. Chang,et al.  Formulation and optimization of cubic polynomial joint trajectories for mechanical manipulators , 1982, 1982 21st IEEE Conference on Decision and Control.

[19]  Judith E. Dayhoff,et al.  Trajectory production with the adaptive time-delay neural network , 1995, Neural Networks.

[20]  Samuel Hunt Drake,et al.  Using compliance in lieu of sensory feedback for automatic assembly. , 1978 .

[21]  Stefan Schaal,et al.  A Generalized Path Integral Control Approach to Reinforcement Learning , 2010, J. Mach. Learn. Res..

[22]  Sethu Vijayakumar,et al.  Learning impedance control of antagonistic systems based on stochastic optimization principles , 2011, Int. J. Robotics Res..

[23]  H. Harry Asada,et al.  Automatic program generation from teaching data for the hybrid control of robots , 1989, IEEE Trans. Robotics Autom..

[24]  Christian Ott,et al.  Cartesian Impedance Control of Redundant and Flexible-Joint Robots , 2008, Springer Tracts in Advanced Robotics.

[25]  Josip Loncaric,et al.  Normal forms of stiffness and compliance matrices , 1987, IEEE Journal on Robotics and Automation.

[26]  Nikolaos G. Tsagarakis,et al.  Statistical dynamical systems for skills acquisition in humanoids , 2012, 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012).

[27]  Sandra Hirche,et al.  Synthesizing Anticipatory Haptic Assistance Considering Human Behavior Uncertainty , 2015, IEEE Transactions on Robotics.

[28]  Barak A. Pearlmutter Learning State Space Trajectories in Recurrent Neural Networks , 1989, Neural Computation.

[29]  Aude Billard,et al.  On Learning, Representing, and Generalizing a Task in a Humanoid Robot , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[30]  Daniel E. Whitney,et al.  Force Feedback Control of Manipulator Fine Motions , 1977 .

[31]  Darwin G. Caldwell,et al.  Learning-based control strategy for safe human-robot interaction exploiting task and robot redundancies , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[32]  Neville Hogan,et al.  Stability properties of human reaching movements , 2004, Experimental Brain Research.

[33]  Roger W. Brockett,et al.  Hybrid Models for Motion Control Systems , 1993 .

[34]  Bruno Siciliano,et al.  Six-DOF impedance control based on angle/axis representations , 1999, IEEE Trans. Robotics Autom..

[35]  T. Lozano-Perez,et al.  Robot programming , 1983, Proceedings of the IEEE.

[36]  Stephen Grossberg,et al.  The Vite Model: A Neural Command Circuit for Generating Arm and Articulator Trajectories, , 1988 .

[37]  Yangsheng Xu,et al.  Online, interactive learning of gestures for human/robot interfaces , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[38]  Giulio Sandini,et al.  An embedded artificial skin for humanoid robots , 2008, 2008 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems.

[39]  Dongheui Lee,et al.  Incremental kinesthetic teaching of motion primitives using the motion refinement tube , 2011, Auton. Robots.

[40]  Sandra Hirche,et al.  Risk-sensitive interaction control in uncertain manipulation tasks , 2013, 2013 IEEE International Conference on Robotics and Automation.

[41]  Oussama Khatib,et al.  A unified approach for motion and force control of robot manipulators: The operational space formulation , 1987, IEEE J. Robotics Autom..

[42]  Daniel E. Koditschek,et al.  Exact robot navigation using artificial potential functions , 1992, IEEE Trans. Robotics Autom..

[43]  Nathan Joseph Delson,et al.  Robot programming by human demonstration , 1994 .

[44]  Stefano Stramigioli,et al.  Modeling and IPC Control of Interactive Mechanical Systems - A Coordinate-Free Approach , 2001 .

[45]  Perry Y. Li,et al.  Passive velocity field control of mechanical manipulators , 1995, IEEE Trans. Robotics Autom..

[46]  Hongbin Wang,et al.  Highly efficient incremental estimation of Gaussian mixture models for online data stream clustering , 2005, SPIE Defense + Commercial Sensing.

[47]  Klaus Neumann,et al.  Neurally imprinted stable vector fields , 2013, ESANN.

[48]  Aude Billard,et al.  Learning Compliant Manipulation through Kinesthetic and Tactile Human-Robot Interaction , 2014, IEEE Transactions on Haptics.

[49]  Pavan Sikka,et al.  Stiffness-based understanding and modeling of contact tasks by human demonstration , 1997, Proceedings of the 1997 IEEE/RSJ International Conference on Intelligent Robot and Systems. Innovative Robotics for Real-World Applications. IROS '97.

[50]  Reza Shadmehr,et al.  Learning the dynamics of reaching movements results in the modification of arm impedance and long-latency perturbation responses , 2001, Biological Cybernetics.

[51]  Daniel E. Whitney,et al.  Historical Perspective and State of the Art in Robot Force Control , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[52]  Alin Albu-Schäffer,et al.  Human-Like Adaptation of Force and Impedance in Stable and Unstable Interactions , 2011, IEEE Transactions on Robotics.

[53]  Nikolaos G. Tsagarakis,et al.  Tele-impedance: Teleoperation with impedance regulation using a body–machine interface , 2012, Int. J. Robotics Res..

[54]  Franco Nardini,et al.  The Dynamical System , 2001 .

[55]  니콜라스 세로프,et al.  Variable impedance control for memory devices , 2012 .

[56]  Klaus Neumann,et al.  Neural learning of stable dynamical systems based on data-driven Lyapunov candidates , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[57]  S. Schaal Dynamic Movement Primitives -A Framework for Motor Control in Humans and Humanoid Robotics , 2006 .

[58]  Russell H. Taylor,et al.  Automatic Synthesis of Fine-Motion Strategies for Robots , 1984 .

[59]  M. Žefran,et al.  A Geometrical Approach to the Study of the Cartesian Stiffness Matrix , 2002 .

[60]  Gerald DeJong,et al.  Explanation-based manipulator learning: Acquisition of planning ability through observation , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[61]  Ales Ude,et al.  Trajectory generation from noisy positions of object features for teaching robot paths , 1993, Robotics Auton. Syst..

[62]  Matthew T. Mason,et al.  Compliance and Force Control for Computer Controlled Manipulators , 1981, IEEE Transactions on Systems, Man, and Cybernetics.

[63]  Stephen P. DeWeerth,et al.  Biologically Inspired Joint Stiffness Control , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[64]  Patrizio Tomei,et al.  On the feedback linearization of robots with elastic joints , 1988, Proceedings of the 27th IEEE Conference on Decision and Control.

[65]  Jean-Claude Latombe,et al.  An Approach to Automatic Robot Programming Based on Inductive Learning , 1984 .

[66]  Zhiwei Luo,et al.  Passive impedance control with time-varying impedance center , 2003, Proceedings 2003 IEEE International Symposium on Computational Intelligence in Robotics and Automation. Computational Intelligence in Robotics and Automation for the New Millennium (Cat. No.03EX694).

[67]  Rieko Osu,et al.  The central nervous system stabilizes unstable dynamics by learning optimal impedance , 2001, Nature.

[68]  Perry Y. Li,et al.  Adaptive passive velocity field control , 1999, Proceedings of the 1999 American Control Conference (Cat. No. 99CH36251).

[69]  Blake Hannaford,et al.  Time domain passivity control of haptic interfaces , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[70]  G. Schöner,et al.  A dynamic theory of coordination of discrete movement , 1990, Biological Cybernetics.

[71]  N. Hogan,et al.  Impedance Control:An Approach to Manipulation,Parts I,II,III , 1985 .