VOR Adaptation on a Humanoid iCub Robot Using a Spiking Cerebellar Model

We embed a spiking cerebellar model within an adaptive real-time (RT) control loop that is able to operate a real robotic body (iCub) when performing different vestibulo-ocular reflex (VOR) tasks. The spiking neural network computation, including event- and time-driven neural dynamics, neural activity, and spike-timing dependent plasticity (STDP) mechanisms, leads to a nondeterministic computation time caused by the neural activity volleys encountered during cerebellar simulation. This nondeterministic computation time motivates the integration of an RT supervisor module that is able to ensure a well-orchestrated neural computation time and robot operation. Actually, our neurorobotic experimental setup (VOR) benefits from the biological sensory motor delay between the cerebellum and the body to buffer the computational overloads as well as providing flexibility in adjusting the neural computation time and RT operation. The RT supervisor module provides for incremental countermeasures that dynamically slow down or speed up the cerebellar simulation by either halting the simulation or disabling certain neural computation features (i.e., STDP mechanisms, spike propagation, and neural updates) to cope with the RT constraints imposed by the real robot operation. This neurorobotic experimental setup is applied to different horizontal and vertical VOR adaptive tasks that are widely used by the neuroscientific community to address cerebellar functioning. We aim to elucidate the manner in which the combination of the cerebellar neural substrate and the distributed plasticity shapes the cerebellar neural activity to mediate motor adaptation. This paper underlies the need for a two-stage learning process to facilitate VOR acquisition.

[1]  Mitsuo Kawato,et al.  Feedback-Error-Learning Neural Network for Supervised Motor Learning , 1990 .

[2]  Mitsuo Kawato,et al.  Statistical characteristics of climbing fiber spikes necessary for efficient cerebellar learning , 2001, Biological Cybernetics.

[3]  Tadashi Yamazaki,et al.  The cerebellum as a liquid state machine , 2007, Neural Networks.

[4]  Thierry Nieus,et al.  A Realistic Large-Scale Model of the Cerebellum Granular Layer Predicts Circuit Spatio-Temporal Filtering Properties , 2009, Front. Cell. Neurosci..

[5]  Tadashi Yamazaki,et al.  Neural Modeling of an Internal Clock , 2005, Neural Computation.

[6]  Eduardo Ros,et al.  Event-Driven Simulation Scheme for Spiking Neural Networks Using Lookup Tables to Characterize Neuronal Dynamics , 2006, Neural Computation.

[7]  D. Schwarz,et al.  Human visuo-vestibular interaction as a basis for quantitative clinical diagnostics. , 1982, Acta oto-laryngologica.

[8]  Wulfram Gerstner,et al.  SPIKING NEURON MODELS Single Neurons , Populations , Plasticity , 2002 .

[9]  Arman Sargolzaei,et al.  Sensorimotor control: computing the immediate future from the delayed present , 2016, BMC Bioinformatics.

[10]  R. Leigh,et al.  The neurology of eye movements , 1984 .

[11]  Tadashi Yamazaki,et al.  Stimulus-Dependent State Transition between Synchronized Oscillation and Randomly Repetitive Burst in a Model Cerebellar Granular Layer , 2011, PLoS Comput. Biol..

[12]  Eduardo Ros,et al.  Real-Time Spiking Neural Network: An Adaptive Cerebellar Model , 2005, IWANN.

[13]  Michael Forrest Biophysics of Purkinje computation , 2008 .

[14]  Egidio D'Angelo,et al.  Distributed cerebellar plasticity implements adaptable gain control in a manipulation task: a closed-loop robotic simulation , 2013, Front. Neural Circuits.

[15]  Stefan Schaal,et al.  Locally Weighted Projection Regression : An O(n) Algorithm for Incremental Real Time Learning in High Dimensional Space , 2000 .

[16]  James V. Stone,et al.  Recurrent cerebellar architecture solves the motor-error problem , 2004, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[17]  R. F. Thompson,et al.  Neural mechanisms of classical conditioning in mammals. , 1990, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[18]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[19]  L. Munari How the body shapes the way we think — a new view of intelligence , 2009 .

[20]  A. Fuchs,et al.  Role of primate flocculus during rapid behavioral modification of vestibuloocular reflex. II. Mossy fiber firing patterns during horizontal head rotation and eye movement. , 1978, Journal of neurophysiology.

[21]  Masao Ito Error detection and representation in the olivo-cerebellar system , 2013, Front. Neural Circuits.

[22]  Alessandra Pedrocchi,et al.  Distributed cerebellar plasticity implements generalized multiple-scale memory components in real-robot sensorimotor tasks , 2015, Front. Comput. Neurosci..

[23]  A. A. Skavenski,et al.  Role of abducens neurons in vestibuloocular reflex. , 1973, Journal of neurophysiology.

[24]  Shigeo Watanabe,et al.  Low-threshold potassium channels and a low-threshold calcium channel regulate Ca2+ spike firing in the dendrites of cerebellar Purkinje neurons: a modeling study , 2001, Brain Research.

[25]  P. Perrin,et al.  How to perform the skull vibration-induced nystagmus test (SVINT). , 2016, European annals of otorhinolaryngology, head and neck diseases.

[26]  Thierry Nieus,et al.  An integrate-and-fire model of a cerebellar granule cell , 2004, Neurocomputing.

[27]  Wolfgang Maass,et al.  Networks of Spiking Neurons: The Third Generation of Neural Network Models , 1996, Electron. Colloquium Comput. Complex..

[28]  Eduardo Ros,et al.  Spike Timing Regulation on the Millisecond Scale by Distributed Synaptic Plasticity at the Cerebellum Input Stage: A Simulation Study , 2013, Front. Comput. Neurosci..

[29]  Giorgio Metta,et al.  YARP: Yet Another Robot Platform , 2006 .

[30]  Eduardo Ros,et al.  From Sensors to Spikes: Evolving Receptive Fields to Enhance Sensorimotor Information in a Robot-Arm , 2012, Int. J. Neural Syst..

[31]  Mitsuo Kawato,et al.  A computational model of four regions of the cerebellum based on feedback-error learning , 2004, Biological Cybernetics.

[32]  Eduardo Ros,et al.  Cerebellar Input Configuration Toward Object Model Abstraction in Manipulation Tasks , 2011, IEEE Transactions on Neural Networks.

[33]  R Angus Silver,et al.  The Contribution of Single Synapses to Sensory Representation in Vivo , 2008, Science.

[34]  Shun-ichi Amari,et al.  A computational study of synaptic mechanisms of partial memory transfer in cerebellar vestibulo-ocular-reflex learning , 2008, Journal of Computational Neuroscience.

[35]  E. W. Kamen,et al.  System identification of the vestibulo-ocular reflex: application of the recursive least-squares algorithm , 1989, Proceedings of the Fifteenth Annual Northeast Bioengineering Conference.

[36]  Jesús Alberto Garrido,et al.  Integrated neural and robotic simulations. Simulation of cerebellar neurobiological substrate for an object-oriented dynamic model abstraction process , 2014, Robotics Auton. Syst..

[37]  Silvia Tolu,et al.  Adaptive and Predictive Control of a Simulated Robot arm , 2013, Int. J. Neural Syst..

[38]  Nicolas Brunel,et al.  A Cerebellar Learning Model of Vestibulo-Ocular Reflex Adaptation in Wild-Type and Mutant Mice , 2014, The Journal of Neuroscience.

[39]  Andrew Howard,et al.  Design and use paradigms for Gazebo, an open-source multi-robot simulator , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[40]  Paolo Dario,et al.  Adaptive gaze stabilization through cerebellar internal models in a humanoid robot , 2016, 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[41]  William Wisden,et al.  Synaptic inhibition of Purkinje cells mediates consolidation of vestibulo-cerebellar motor learning , 2009, Nature Neuroscience.

[42]  Stefan Schaal,et al.  Biomimetic gaze stabilization based on feedback-error-learning with nonparametric regression networks , 2001, Neural Networks.

[43]  Silvia Tolu,et al.  Bio-inspired adaptive feedback error learning architecture for motor control , 2012, Biological Cybernetics.

[44]  Mancia Anguita,et al.  A Spiking Neural Simulator Integrating Event-Driven and Time-Driven Computation Schemes Using Parallel CPU-GPU Co-Processing: A Case Study , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[45]  Eduardo Ros,et al.  A real-time spiking cerebellum model for learning robot control , 2008, Biosyst..

[46]  J. M. Fredrickson,et al.  Quantification of the vestibulo-ocular reflex and visual-vestibular interaction for the purpose of clinical diagnosis , 2006, Medical and Biological Engineering and Computing.

[47]  Eduardo Ros,et al.  Cerebellarlike Corrective Model Inference Engine for Manipulation Tasks , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[48]  K. Doya,et al.  Electrophysiological properties of inferior olive neurons: A compartmental model. , 1999, Journal of neurophysiology.

[49]  M. Fujita,et al.  Adaptive filter model of the cerebellum , 1982, Biological Cybernetics.

[50]  Richard D. Rabbitt,et al.  Biomechanics of the Semicircular Canals and Otolith Organs , 2004 .

[51]  Eduardo Ros,et al.  Fast convergence of learning requires plasticity between inferior olive and deep cerebellar nuclei in a manipulation task: a closed-loop robotic simulation , 2014, Front. Comput. Neurosci..

[52]  Eduardo Ros,et al.  Event- and Time-Driven Techniques Using Parallel CPU-GPU Co-processing for Spiking Neural Networks , 2017, Front. Neuroinform..

[53]  Roger D. Traub,et al.  High-Frequency Network Oscillations in Cerebellar Cortex , 2008, Neuron.

[54]  Silvia Tolu,et al.  Oscillation-Driven Spike-Timing Dependent Plasticity Allows Multiple Overlapping Pattern Recognition in Inhibitory Interneuron Networks , 2016, Int. J. Neural Syst..

[55]  Pablo Varona,et al.  Transient dynamics and rhythm coordination of inferior olive spatio-temporal patterns , 2013, Front. Neural Circuits.

[56]  Wofgang Maas,et al.  Networks of spiking neurons: the third generation of neural network models , 1997 .

[57]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1952, The Journal of physiology.

[58]  D A Robinson,et al.  The use of control systems analysis in the neurophysiology of eye movements. , 1981, Annual review of neuroscience.

[59]  S G Lisberger,et al.  Partial ablations of the flocculus and ventral paraflocculus in monkeys cause linked deficits in smooth pursuit eye movements and adaptive modification of the VOR. , 2002, Journal of neurophysiology.

[60]  Nikolaos G. Tsagarakis,et al.  iCub: the design and realization of an open humanoid platform for cognitive and neuroscience research , 2007, Adv. Robotics.

[61]  Rolf Pfeifer,et al.  How the body shapes the way we think - a new view on intelligence , 2006 .

[62]  Eduardo Ros,et al.  Distributed Cerebellar Motor Learning: A Spike-Timing-Dependent Plasticity Model , 2016, Front. Comput. Neurosci..

[63]  Silvia Tolu,et al.  Adaptive cerebellar Spiking Model Embedded in the Control Loop: Context Switching and Robustness against noise , 2011, Int. J. Neural Syst..

[64]  Shigeru Tanaka,et al.  Computational Models of Timing Mechanisms in the Cerebellar Granular Layer , 2009, The Cerebellum.