Evidence for Composite Cost Functions in Arm Movement Planning: An Inverse Optimal Control Approach

An important issue in motor control is understanding the basic principles underlying the accomplishment of natural movements. According to optimal control theory, the problem can be stated in these terms: what cost function do we optimize to coordinate the many more degrees of freedom than necessary to fulfill a specific motor goal? This question has not received a final answer yet, since what is optimized partly depends on the requirements of the task. Many cost functions were proposed in the past, and most of them were found to be in agreement with experimental data. Therefore, the actual principles on which the brain relies to achieve a certain motor behavior are still unclear. Existing results might suggest that movements are not the results of the minimization of single but rather of composite cost functions. In order to better clarify this last point, we consider an innovative experimental paradigm characterized by arm reaching with target redundancy. Within this framework, we make use of an inverse optimal control technique to automatically infer the (combination of) optimality criteria that best fit the experimental data. Results show that the subjects exhibited a consistent behavior during each experimental condition, even though the target point was not prescribed in advance. Inverse and direct optimal control together reveal that the average arm trajectories were best replicated when optimizing the combination of two cost functions, nominally a mix between the absolute work of torques and the integrated squared joint acceleration. Our results thus support the cost combination hypothesis and demonstrate that the recorded movements were closely linked to the combination of two complementary functions related to mechanical energy expenditure and joint-level smoothness.

[1]  Sylvain Calino,et al.  Robot programming by demonstration : a probabilistic approach , 2009 .

[2]  宇野 洋二,et al.  Formation and control of optimal trajectory in human multijoint arm movement : minimum torque-change model , 1988 .

[3]  L. S. Pontryagin,et al.  Mathematical Theory of Optimal Processes , 1962 .

[4]  Rajesh P. N. Rao,et al.  Bayesian brain : probabilistic approaches to neural coding , 2006 .

[5]  Aude Billard,et al.  On Learning, Representing, and Generalizing a Task in a Humanoid Robot , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[6]  Vladimir M. Zatsiorsky,et al.  Analytical and numerical analysis of inverse optimization problems: conditions of uniqueness and computational methods , 2011, Biological Cybernetics.

[7]  Emmanuel Guigon,et al.  Computational Motor Control : Redundancy and Invariance , 2007 .

[8]  Paul L Gribble,et al.  Role of cocontraction in arm movement accuracy. , 2003, Journal of neurophysiology.

[9]  M. Flanders,et al.  Arm movements in three-dimensional space: computation, theory, and observation. , 1991, Exercise and sport sciences reviews.

[10]  Michael I. Jordan,et al.  Constrained and unconstrained movements involve different control strategies. , 1997, Journal of neurophysiology.

[11]  Stan Gielen,et al.  Review of models for the generation of multi-joint movements in 3-D. , 2009, Advances in experimental medicine and biology.

[12]  R. E. Kalman,et al.  When Is a Linear Control System Optimal , 1964 .

[13]  M. J. D. Powell,et al.  UOBYQA: unconstrained optimization by quadratic approximation , 2002, Math. Program..

[14]  T. Flash,et al.  The coordination of arm movements: an experimentally confirmed mathematical model , 1985, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[15]  Michael I. Jordan,et al.  Smoothness maximization along a predefined path accurately predicts the speed profiles of complex arm movements. , 1998, Journal of neurophysiology.

[16]  Daniel M. Wolpert,et al.  Signal-dependent noise determines motor planning , 1998, Nature.

[17]  G. Zajicek,et al.  The Wisdom of the Body , 1934, Nature.

[18]  Sascha E. Engelbrecht,et al.  Minimum Principles in Motor Control. , 2001, Journal of mathematical psychology.

[19]  M. L. Chambers The Mathematical Theory of Optimal Processes , 1965 .

[20]  M. Kawato,et al.  Formation and control of optimal trajectory in human multijoint arm movement , 1989, Biological Cybernetics.

[21]  Konrad Paul Körding,et al.  The loss function of sensorimotor learning. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[22]  D. Rosenbaum,et al.  Planning reaching and grasping movements: theoretical premises and practical implications. , 2001, Motor control.

[23]  T. Flash,et al.  Comparing Smooth Arm Movements with the Two-Thirds Power Law and the Related Segmented-Control Hypothesis , 2002, The Journal of Neuroscience.

[24]  D. Pélisson,et al.  From Eye to Hand: Planning Goal-directed Movements , 1998, Neuroscience & Biobehavioral Reviews.

[25]  J. McIntyre,et al.  Kinematic and dynamic processes for the control of pointing movements in humans revealed by short-term exposure to microgravity , 2005, Neuroscience.

[26]  M. Landy,et al.  Decision making, movement planning and statistical decision theory , 2008, Trends in Cognitive Sciences.

[27]  Sylvain Calinon,et al.  Robot Programming by Demonstration - a Probabilistic Approach , 2009 .

[28]  J. Gauthier,et al.  A biomechanical inactivation principle , 2010 .

[29]  Richard M. Murray,et al.  A Mathematical Introduction to Robotic Manipulation , 1994 .

[30]  C. Atkeson,et al.  Kinematic features of unrestrained vertical arm movements , 1985, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[31]  Michael I. Jordan,et al.  Optimal feedback control as a theory of motor coordination , 2002, Nature Neuroscience.

[32]  Andrew Y. Ng,et al.  Pharmacokinetics of a novel formulation of ivermectin after administration to goats , 2000, ICML.

[33]  H. Cruse,et al.  The human arm as a redundant manipulator: The control of path and joint angles , 2004, Biological Cybernetics.

[34]  Amir Karniel,et al.  Minimum Acceleration Criterion with Constraints Implies Bang-Bang Control as an Underlying Principle for Optimal Trajectories of Arm Reaching Movements , 2008, Neural Computation.

[35]  Konrad Paul Kording,et al.  A Neuroeconomics Approach to Inferring Utility Functions in Sensorimotor Control , 2004, PLoS biology.

[36]  Denis Mottet,et al.  The role of cocontraction in the impairment of movement accuracy with fatigue , 2008, Experimental Brain Research.

[37]  J. F. Soechting,et al.  Coordination of arm and wrist motion during a reaching task , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[38]  John E. R. Staddon,et al.  Optima for animals , 1982 .

[39]  Loukia D. Loukopoulos,et al.  Planning reaches by evaluating stored postures. , 1995, Psychological review.

[40]  T. Pozzo,et al.  Trajectories of arm pointing movements on the sagittal plane vary with both direction and speed , 2003, Experimental Brain Research.

[41]  Anil V. Rao,et al.  Direct Trajectory Optimization and Costate Estimation via an Orthogonal Collocation Method , 2006 .

[42]  Philippe Lefèvre,et al.  Optimal integration of gravity in trajectory planning of vertical pointing movements. , 2009, Journal of neurophysiology.

[43]  Stephen P. Boyd,et al.  Linear Matrix Inequalities in Systems and Control Theory , 1994 .

[44]  E. Yaz Linear Matrix Inequalities In System And Control Theory , 1998, Proceedings of the IEEE.

[45]  Jean-Paul Gauthier,et al.  The Inactivation Principle: Mathematical Solutions Minimizing the Absolute Work and Biological Implications for the Planning of Arm Movements , 2008, PLoS Comput. Biol..

[46]  Armin Biess,et al.  A Computational Model for Redundant Human Three-Dimensional Pointing Movements: Integration of Independent Spatial and Temporal Motor Plans Simplifies Movement Dynamics , 2007, The Journal of Neuroscience.

[47]  Emanuel Todorov,et al.  From task parameters to motor synergies: A hierarchical framework for approximately optimal control of redundant manipulators , 2005, J. Field Robotics.

[48]  Frans C. T. van der Helm,et al.  Musculoskeletal Systems with Intrinsic and Proprioceptive Feedback , 2000 .

[49]  William W. Hager,et al.  A unified framework for the numerical solution of optimal control problems using pseudospectral methods , 2010, Autom..

[50]  E. Todorov Optimality principles in sensorimotor control , 2004, Nature Neuroscience.

[51]  Li Tiancheng,et al.  アルゴリズム906: elrint3d―組み込み格子ルールのシーケンスを用いる三次元非適応自動立体求積法ルーチン , 2011 .

[52]  Konrad Paul Kording,et al.  Decision Theory: What "Should" the Nervous System Do? , 2007, Science.

[53]  P. Viviani,et al.  The law relating the kinematic and figural aspects of drawing movements. , 1983, Acta psychologica.

[54]  W. L. Nelson Physical principles for economies of skilled movements , 1983, Biological Cybernetics.

[55]  Jun Nishii,et al.  Legged insects select the optimal locomotor pattern based on the energetic cost , 2000, Biological Cybernetics.

[56]  Les S. Jennings,et al.  Performance objectives in human movement: A review and application to the stance phase of normal walking , 1989 .

[57]  Y Uno,et al.  Quantitative examinations of internal representations for arm trajectory planning: minimum commanded torque change model. , 1999, Journal of neurophysiology.

[58]  M. Shadlen,et al.  Representation of Confidence Associated with a Decision by Neurons in the Parietal Cortex , 2009, Science.

[59]  J. F. Soechting,et al.  Invariant characteristics of a pointing movement in man , 1981, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[60]  Claude Bernard,et al.  Leçons sur les phénomènes de la vie communs aux animaux et aux végétaux , 1878 .

[61]  J. F. Soechting,et al.  Moving effortlessly in three dimensions: does Donders' law apply to arm movement? , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[62]  Jean-Paul Laumond,et al.  From human to humanoid locomotion—an inverse optimal control approach , 2010, Auton. Robots.

[63]  Frank Vanden Berghen,et al.  CONDOR, a new parallel, constrained extension of Powell's UOBYQA algorithm: experimental results and comparison with the DFO algorithm , 2005 .

[64]  Emanuel Todorov,et al.  Evidence for the Flexible Sensorimotor Strategies Predicted by Optimal Feedback Control , 2007, The Journal of Neuroscience.

[65]  Rieko Osu,et al.  Quantitative examinations for multi joint arm trajectory planning--using a robust calculation algorithm of the minimum commanded torque change trajectory , 2001, Neural Networks.

[66]  T. Flash,et al.  Riemannian geometric approach to human arm dynamics, movement optimization, and invariance. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[67]  Mark L Latash,et al.  An analytical approach to the problem of inverse optimization with additive objective functions: an application to human prehension , 2010, Journal of mathematical biology.

[68]  Michael A. Saunders,et al.  SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization , 2005, SIAM Rev..

[69]  F. Clarke Optimization And Nonsmooth Analysis , 1983 .

[70]  C. R. Taylor,et al.  Principles of Animal Design: The Optimization And Symmorphosis Debate , 1998 .

[71]  Emanuel Todorov,et al.  Optimal Control Theory , 2006 .

[72]  Anil V. Rao,et al.  Algorithm 902: GPOPS, A MATLAB software for solving multiple-phase optimal control problems using the gauss pseudospectral method , 2010, TOMS.

[73]  D A Rosenbaum,et al.  Cooperative selection of movements: The optimal selection model , 1996, Psychological research.

[74]  Michael A. Saunders,et al.  SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization , 2002, SIAM J. Optim..

[75]  Pat Langley,et al.  Editorial: On Machine Learning , 1986, Machine Learning.