Dynamic Movement Primitives -A Framework for Motor Control in Humans and Humanoid Robotics

Given the continuous stream of movements that biological systems exhibit in their daily activities, an account for such versatility and creativity has to assume that movement sequences consist of segments, executed either in sequence or with partial or complete overlap. Therefore, a fundamental question that has pervaded research in motor control both in artificial and biological systems revolves around identifying movement primitives (a.k.a. units of actions, basis behaviors, motor schemas, etc.). What are the fundamental building blocks that are strung together, adapted to, and created for ever new behaviors? This paper summarizes results that led to the hypothesis of Dynamic Movement Primitives (DMP). DMPs are units of action that are formalized as stable nonlinear attractor systems. They are useful for autonomous robotics as they are highly flexible in creating complex rhythmic (e.g., locomotion) and discrete (e.g., a tennis swing) behaviors that can quickly be adapted to the inevitable perturbations of a dynamically changing, stochastic environment. Moreover, DMPs provide a formal framework that also lends itself to investigations in computational neuroscience. A recent finding that allows creating DMPs with the help of well-understood statistical learning methods has elevated DMPs from a more heuristic to a principled modeling approach. Theoretical insights, evaluations on a humanoid robot, and behavioral and brain imaging data will serve to outline the framework of DMPs for a general approach to motor control in robotics and biology.

[1]  R. Bellman Dynamic programming. , 1957, Science.

[2]  M. Ciletti,et al.  The computation and theory of optimal control , 1972 .

[3]  C. Lent On neuronal nihilism , 1980, Behavioral and Brain Sciences.

[4]  Allen I. Selverston,et al.  Are central pattern generators understandable? , 1980, Behavioral and Brain Sciences.

[5]  P. Viviani,et al.  32 Space-Time Invariance in Learned Motor Skills , 1980 .

[6]  G. Stelmach,et al.  Tutorials in Motor Behavior , 1980 .

[7]  Michael A. Arbib,et al.  Perceptual Structures and Distributed Motor Control , 1981 .

[8]  P. Viviani,et al.  The law relating the kinematic and figural aspects of drawing movements. , 1983, Acta psychologica.

[9]  J. Hollerbach Dynamic Scaling of Manipulator Trajectories , 1983, 1983 American Control Conference.

[10]  P Viviani,et al.  Segmentation and coupling in complex movements. , 1985, Journal of experimental psychology. Human perception and performance.

[11]  Paolo Viviani,et al.  Do Units of Motor Action Really Exist , 1986 .

[12]  Rodney A. Brooks,et al.  A Robust Layered Control Syste For A Mobile Robot , 2022 .

[13]  Marc H. Raibert,et al.  Legged Robots That Balance , 1986, IEEE Expert.

[14]  M. T. Turveyt The Challenge of a Physical Account of Action: A Personal View* , 1987 .

[15]  Daniel E. Koditschek,et al.  Exact robot navigation by means of potential functions: Some topological considerations , 1987, Proceedings. 1987 IEEE International Conference on Robotics and Automation.

[16]  J. Soechting,et al.  Organization of arm movements in three-dimensional space. Wrist motion is piecewise planar , 1987, Neuroscience.

[17]  J. F. Soechting,et al.  Organization of arm movements. Motion is segmented , 1987, Neuroscience.

[18]  A. Wing,et al.  Relation between velocity and curvature in movement: equivalence and divergence between a power law and a minimum-jerk model. , 1988, Journal of experimental psychology. Human perception and performance.

[19]  Christopher G. Atkeson,et al.  Task-level robot learning: juggling a tennis ball more accurately , 1989, Proceedings, 1989 International Conference on Robotics and Automation.

[20]  Daniel E. Koditschek,et al.  Robotic tasks with intermittent dynamics , 1990 .

[21]  Phillip J. McKerrow,et al.  Introduction to robotics , 1991 .

[22]  Gerald Tesauro,et al.  Temporal Difference Learning of Backgammon Strategy , 1992, ML Workshop.

[23]  Stefan Schaal,et al.  Open loop stable control strategies for robot juggling , 1993, [1993] Proceedings IEEE International Conference on Robotics and Automation.

[24]  Daniel E. Koditschek,et al.  Further progress in robot juggling: solvable mirror laws , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[25]  A. Opstal Dynamic Patterns: The Self-Organization of Brain and Behavior , 1995 .

[26]  T. Flash,et al.  Minimum-jerk, two-thirds power law, and isochrony: converging approaches to movement planning. , 1995, Journal of experimental psychology. Human perception and performance.

[27]  D. Ostry,et al.  Origins of the power law relation between movement velocity and curvature: modeling the effects of muscle mechanics and limb dynamics. , 1996, Journal of neurophysiology.

[28]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[29]  S. Schaal,et al.  One-Handed Juggling: A Dynamical Approach to a Rhythmic Movement Task. , 1996, Journal of motor behavior.

[30]  S Grossberg,et al.  Neural control of interlimb oscillations. I. Human bimanual coordination. , 1997, Biological cybernetics.

[31]  E. Bizzi,et al.  Learning Newtonian Mechanics , 1997 .

[32]  S Grossberg,et al.  Neural control of interlimb oscillations. II. Biped and quadruped gaits and bifurcations. , 1997, Biological cybernetics.

[33]  M. Arbib,et al.  Role of the cerebellum in reaching movements in humans. I. Distributed inverse dynamics control , 1998, The European journal of neuroscience.

[34]  M. Turvey,et al.  Interlimb coupling in a simple serial behavior: A task dynamic approach , 1998 .

[35]  Christopher G. Atkeson,et al.  Constructive Incremental Learning from Only Local Information , 1998, Neural Computation.

[36]  Jean-Jacques E. Slotine,et al.  On Contraction Analysis for Non-linear Systems , 1998, Autom..

[37]  S. Schaal,et al.  Programmable Pattern Generators , 1998 .

[38]  M. Arbib,et al.  Role of the cerebellum in reaching movements in humans. II. A neural model of the intermediate cerebellum , 1998, The European journal of neuroscience.

[39]  Matthew M. Williamson,et al.  Neural control of rhythmic arm movements , 1998, Neural Networks.

[40]  Daniel E. Koditschek,et al.  Sequential Composition of Dynamically Dexterous Robot Behaviors , 1999, Int. J. Robotics Res..

[41]  S. Schaal,et al.  Segmentation of endpoint trajectories does not imply segmented control , 1999, Experimental Brain Research.

[42]  Winfried Stefan Lohmiller,et al.  Contraction analysis of nonlinear systems , 1999 .

[43]  Stefan Schaal,et al.  Is imitation learning the route to humanoid robots? , 1999, Trends in Cognitive Sciences.

[44]  E. Marder Motor pattern generation , 2000, Current Opinion in Neurobiology.

[45]  S. Schaal,et al.  Origins and violations of the 2/3 power law in rhythmic three-dimensional arm movements , 2000, Experimental Brain Research.

[46]  P. Strick,et al.  Imaging the premotor areas , 2001, Current Opinion in Neurobiology.

[47]  Dagmar Sternad,et al.  Origins and Violations of the 2/3 Power Law in Rhythmic 3D Arm Movements , 2001 .

[48]  Jun Nakanishi,et al.  Learning Attractor Landscapes for Learning Motor Primitives , 2002, NIPS.

[49]  William J. Dean,et al.  Interaction of discrete and rhythmic movements over a wide range of periods , 2002, Experimental Brain Research.

[50]  P. Morasso Three dimensional arm trajectories , 1983, Biological Cybernetics.

[51]  Hiroshi Shimizu,et al.  Self-organized control of bipedal locomotion by neural oscillators in unpredictable environment , 1991, Biological Cybernetics.

[52]  A. G. Feldman,et al.  Merging different motor patterns: Coordination between rhythmical and discrete single-joint movements , 2004, Experimental Brain Research.

[53]  G. Schöner,et al.  A dynamic theory of coordination of discrete movement , 1990, Biological Cybernetics.

[54]  Apostolos P. Georgopoulos,et al.  Three-dimensional drawings in isometric conditions: planar segmentation of force trajectory , 1992, Experimental Brain Research.

[55]  Michael T. Turvey,et al.  Average phase difference theory and 1∶1 phase entrainment in interlimb coordination , 1992, Biological Cybernetics.

[56]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[57]  M. Mon-Williams,et al.  Motor Control and Learning , 2006 .