Hierarchical control by a higher center and the rhythm generator contributes to realize adaptive locomotion

Many cyclic movements of vertebrates are produced by a rhythm generator in the spinal cord, which is often referred to as the central pattern generator. Meanwhile, higher centers, such as the cerebellum, are also involved in motor control. In this study, we discuss the control and learning mechanisms of these two control systems, focusing on the following problems: (1) how these two control systems generate a motor command cooperatively without conflict, and (2) how these control systems acquire motor commands. As a possible model to solve these problems, we propose a hierarchical motor control learning model on the basis of physiological knowledge. This model assumes that the higher center learns the timing and amplitude of a motor command and sends control signals to the rhythm generator. The rhythm generator is tuned by the control signals and acquires the ability to output a motor command to realize a target motion. By applying this model to a simulation experiment of the learning control of a one-dimensional hopping robot, hopping at a desired height was successful. Our simulation results also show that a multiple control system involving the higher center and rhythm generator is more robust than a single control system.

[1]  J. Kalaska,et al.  Muscle synergies during locomotion in the cat: a model for motor cortex control , 2008, The Journal of physiology.

[2]  Vijaykumar Gullapalli,et al.  A stochastic reinforcement learning algorithm for learning real-valued functions , 1990, Neural Networks.

[3]  Jun Nishii,et al.  Constraint and exploitation of redundant degrees of freedom during walking , 2012, Robotics Auton. Syst..

[4]  Jun Nishii,et al.  A learning model for oscillatory networks , 1998, Neural Networks.

[5]  M. Udo,et al.  Climbing fiber responses in cerebellar vermal Purkinje cells during perturbed locomotion in decerebrate cats , 1994, Neuroscience Research.

[6]  R. Kram,et al.  Mechanical and metabolic determinants of the preferred step width in human walking , 2001, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[7]  S. Grillner,et al.  The role of spinal cord inputs in modulating the activity of reticulospinal neurons during fictive locomotion in the lamprey , 1989, Brain Research.

[8]  S. Grillner.  Neurobiological bases of rhythmic motor acts in vertebrates. , 1985, Science.

[9]  O. I. Fukson,et al.  Recordings of neurones of the dorsal spinocerebellar tract during evoked locomotion. , 1972, Brain research.

[10]  Jun Nishii,et al.  Legged insects select the optimal locomotor pattern based on the energetic cost , 2000, Biological Cybernetics.

[11]  A. Minetti,et al.  A theory of metabolic costs for bipedal gaits. , 1997, Journal of theoretical biology.

[12]  Serge Rossignol,et al.  Prominent Role of the Spinal Central Pattern Generator in the Recovery of Locomotion after Partial Spinal Cord Injuries , 2008, The Journal of Neuroscience.

[13]  S. Grillner,et al.  Peripheral control of the cat's step cycle. II. Entrainment of the central pattern generators for locomotion by sinusoidal hip movements during "fictive locomotion.". , 1983, Acta physiologica Scandinavica.

[14]  J. Nishii.  An analytical estimation of the energy cost for legged locomotion. , 2006, Journal of theoretical biology.

[15]  O. I. Fukson,et al.  [Activity of neurons of the dorsal spinocerebellar tract during locomotion]. , 1972, Biofizika.

[16]  Jun Nishii.  A Learning Model of a Periodic Locomotor Pattern by the Central Pattern Generator , 1999, Adapt. Behav..

[17]  C. W. Radcliffe,et al.  Predicting metabolic cost of level walking , 1978, European Journal of Applied Physiology and Occupational Physiology.

[18]  Shik Ml,et al.  Control of walking and running by means of electric stimulation of the midbrain , 1966 .

[19]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[20]  S. Rossignol,et al.  Microstimulation of the medullary reticular formation during fictive locomotion. , 1994, Journal of neurophysiology.

[21]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[22]  K. Pearson,et al.  Computer simulation of stepping in the hind legs of the cat: an examination of mechanisms regulating the stance-to-swing transition. , 2005, Journal of neurophysiology.

[23]  K. Sigvardt,et al.  Features of entrainment of spinal pattern generators for locomotor activity in the lamprey spinal cord , 1988, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[24]  K. Saitoh,et al.  Role of basal ganglia–brainstem pathways in the control of motor behaviors , 2004, Neuroscience Research.

[25]  K. Saitoh,et al.  Basal ganglia efferents to the brainstem centers controlling postural muscle tone and locomotion: a new concept for understanding motor disorders in basal ganglia dysfunction , 2003, Neuroscience.

[26]  Ryoji Suzuki,et al.  Mathematical models for the swimming pattern of a lamprey , 1994, Biological Cybernetics.

[27]  Shinya Aoi,et al.  Evaluating functional roles of phase resetting in generation of adaptive human bipedal walking with a physiologically based model of the spinal pattern generator , 2010, Biological Cybernetics.

[28]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[29]  Ryoji Suzuki,et al.  Mathematical models for the swimming pattern of a lamprey , 1994, Biological Cybernetics.

[30]  Hiroshi Nishimaru,et al.  Formation of the central pattern generator for locomotion in the rat and mouse , 2000, Brain Research Bulletin.

[31]  S. Grillner,et al.  Neuronal network generating locomotor behavior in lamprey: circuitry, transmitters, membrane properties, and simulation. , 1991, Annual review of neuroscience.

[32]  O. I. Fukson,et al.  Origin of modulation in neurones of the ventral spinocerebellar tract during locomotion. , 1972, Brain research.

[33]  D J Kriellaars,et al.  Mechanical entrainment of fictive locomotion in the decerebrate cat. , 1994, Journal of neurophysiology.

[34]  C. Rovainen.  Physiological and anatomical studies on large neurons of central nervous system of the sea lamprey (Petromyzon marinus). I. Müller and Mauthner cells. , 1967, Journal of neurophysiology.