Machine Learning Capabilities of a Simulated Cerebellum

This paper describes the learning and control capabilities of a biologically constrained bottom-up model of the mammalian cerebellum. Results are presented from six tasks: 1) eyelid conditioning; 2) pendulum balancing; 3) proportional–integral–derivative control; 4) robot balancing; 5) pattern recognition; and 6) MNIST handwritten digit recognition. These tasks span several paradigms of machine learning, including supervised learning, reinforcement learning, control, and pattern recognition. Results over these six domains indicate that the cerebellar simulation is capable of robustly identifying static input patterns even when randomized across the sensory apparatus. This capability allows the simulated cerebellum to perform several different supervised learning and control tasks. On the other hand, both reinforcement learning and temporal pattern recognition prove problematic due to the delayed nature of error signals and the simulator’s inability to solve the credit assignment problem. These results are consistent with previous findings which hypothesize that in the human brain, the basal ganglia is responsible for reinforcement learning, while the cerebellum handles supervised learning.

[1]  N. Donegan,et al.  A model of Pavlovian eyelid conditioning based on the synaptic organization of the cerebellum. , 1997, Learning & memory.

[2]  K. Doya.  Complementary roles of basal ganglia and cerebellum in learning and motor control , 2000, Current Opinion in Neurobiology.

[3]  R. F. Thompson,et al.  Cerebellum: essential involvement in the classically conditioned eyelid response. , 1984, Science.

[4]  Kenji Doya,et al.  What are the computations of the cerebellum, the basal ganglia and the cerebral cortex? , 1999, Neural Networks.

[5]  Dean V. Buonomano,et al.  Neural Network Model of the Cerebellum: Temporal Discrimination and the Timing of Motor Responses , 1999, Neural Computation.

[6]  G. Shepherd.  The Synaptic Organization of the Brain , 1979 .

[7]  P. Verschure,et al.  The cerebellum in action: a simulation and robotics study , 2002, The European journal of neuroscience.

[8]  Jing Chen,et al.  On-line NNAC for a Balancing Two-Wheeled Robot Using Feedback-Error-Learning on the Neurophysiological Mechanism , 2011, J. Comput..

[9]  J. Albus.  A Theory of Cerebellar Function , 1971 .

[10]  Javier F. Medina,et al.  Timing Mechanisms in the Cerebellum: Testing Predictions of a Large-Scale Computer Simulation , 2000, The Journal of Neuroscience.

[11]  R.M. Dunn,et al.  Brains, behavior, and robotics , 1983, Proceedings of the IEEE.

[12]  A. Barto,et al.  Models of the cerebellum and motor learning , 1996 .

[13]  Peter Stone,et al.  RTMBA: A Real-Time Model-Based Reinforcement Learning Architecture for robot control , 2011, 2012 IEEE International Conference on Robotics and Automation.

[14]  Tadashi Yamazaki,et al.  Realtime cerebellum: A large-scale spiking network model of the cerebellum that runs in realtime using a graphics processing unit , 2013, Neural Networks.

[15]  D. Wolpert,et al.  Internal models in the cerebellum , 1998, Trends in Cognitive Sciences.

[16]  C.W. Anderson,et al.  Learning to control an inverted pendulum using neural networks , 1989, IEEE Control Systems Magazine.

[17]  伊藤 正男.  The cerebellum and neural control , 1984 .

[18]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[19]  Peter Stone,et al.  Using a million cell simulation of the cerebellum: Network scaling and task generality , 2013, Neural Networks.

[20]  John C. Eccles,et al.  The Mossy Fiber Input into the Cerebellar Cortex and its Inhibitory Control by Golgi Cells , 1967 .

[21]  M. Glickstein,et al.  The anatomy of the cerebellum , 1998, Trends in Neurosciences.

[22]  J. Houk.  Cooperative Control of Limb Movements by the Motor Cortex, Brainstem and , 1989 .

[23]  A. Pellionisz,et al.  Tensor network theory of the metaorganization of functional geometries in the central nervous system , 1985, Neuroscience.

[24]  A G Barto,et al.  Prediction of complex two-dimensional trajectories by a cerebellar model of smooth pursuit eye movement. , 1997, Journal of neurophysiology.

[25]  Wen-Ke Li,et al.  Timing in the cerebellum : a matter of network inhibition , 2015 .

[26]  M. Mauk,et al.  What the cerebellum computes , 2003, Trends in Neurosciences.

[27]  Peter Stone,et al.  TEXPLORE: real-time sample-efficient reinforcement learning for robots , 2012, Machine Learning.

[28]  M. Kawato,et al.  The cerebellum and VOR/OKR learning models , 1992, Trends in Neurosciences.

[29]  W. Skaggs,et al.  The Cerebellum , 2016 .

[30]  Gerald M Edelman,et al.  A cerebellar model for predictive motor control tested in a brain-based device. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[31]  D. Perkel,et al.  A computer model of the cerebellar cortex of the frog , 1977, Neuroscience.

[32]  Michael D Mauk,et al.  A Subtraction Mechanism of Temporal Coding in Cerebellar Cortex , 2011, The Journal of Neuroscience.

[33]  Peter Stone,et al.  Multiagent interactions in urban driving , 2008 .

[34]  D. Marr.  A theory of cerebellar cortex , 1969, The Journal of physiology.

[35]  Yun Li,et al.  PID control system analysis, design, and technology , 2005, IEEE Transactions on Control Systems Technology.

[36]  Y. Prigent.  [Long term depression]. , 1989, Annales medico-psychologiques.

[37]  Professor Dr. John C. Eccles,et al.  The Cerebellum as a Neuronal Machine , 1967, Springer Berlin Heidelberg.

[38]  Javier F. Medina,et al.  Computer simulation of cerebellar information processing , 2000, Nature Neuroscience.

[39]  Silvia Tolu,et al.  Adaptive cerebellar Spiking Model Embedded in the Control Loop: Context Switching and Robustness against noise , 2011, Int. J. Neural Syst..

[40]  L. Baum,et al.  Statistical Inference for Probabilistic Functions of Finite State Markov Chains , 1966 .

[41]  Jeffrey L. Krichmar,et al.  Embodied models of delayed neural responses: Spatiotemporal categorization and predictive motor control in brain based devices , 2008, Neural Networks.

[42]  Ling Xu,et al.  Cerebellar dynamic state estimation for a biomorphic robot arm , 2005, 2005 IEEE International Conference on Systems, Man and Cybernetics.

[43]  M. Alexander,et al.  Principles of Neural Science , 1981 .

[44]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[45]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[46]  G. Kenyon,et al.  A model of long-term memory storage in the cerebellar cortex: a possible role for plasticity at parallel fiber synapses onto stellate/basket interneurons. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[47]  W T Thach,et al.  The cerebellum and the adaptive coordination of movement. , 1992, Annual review of neuroscience.

[48]  A. Pellionisz,et al.  Tensorial approach to the geometry of brain function: Cerebellar coordination via a metric tensor , 1980, Neuroscience.

[49]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.