A Biomimetic Control Method Increases the Adaptability of a Humanoid Robot Acting in a Dynamic Environment

One of the big challenges in robotics is to endow agents with autonomous and adaptive capabilities. With this purpose, we embedded a cerebellum-based control system into a humanoid robot that becomes capable of handling dynamical external and internal complexity. The cerebellum is the area of the brain that coordinates and predicts the body movements throughout the body-environment interactions. Different biologically plausible cerebellar models are available in literature and have been employed for motor learning and control of simplified objects. We built the canonical cerebellar microcircuit by combining machine learning and computational neuroscience techniques. The control system is composed of the adaptive cerebellar module and a classic control method; their combination allows a fast adaptive learning and robust control of the robotic movements when external disturbances appear. The control structure is built offline, but the dynamic parameters are learned during an online-phase training. The aforementioned adaptive control system has been tested in the Neuro-robotics Platform with the virtual humanoid robot iCub. In the experiment, the robot iCub has to balance with the hand a table with a ball running on it. In contrast with previous attempts of solving this task, the proposed neural controller resulted able to quickly adapt when the internal and external conditions change. Our bio-inspired and flexible control architecture can be applied to different robotic configurations without an excessive tuning of the parameters or customization. The cerebellum-based control system is indeed able to deal with changing dynamics and interactions with the environment. Important insights regarding the relationship between the bio-inspired control system functioning and the complexity of the task to be performed are obtained.

[1]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[2]  Bruce A. Francis,et al.  The internal model principle of control theory , 1976, Autom..

[3]  Juyang Weng,et al.  A theory for mentally developing robots , 2002, Proceedings 2nd International Conference on Development and Learning. ICDL 2002.

[4]  Chenguang Yang,et al.  Bioinspired control design using cerebellar model articulation controller network for omnidirectional mobile robots , 2018 .

[5]  Jun Nakanishi,et al.  Feedback error learning and nonlinear adaptive control , 2004, Neural Networks.

[6]  James C. Houk,et al.  A Cerebellar Model of Timing and Prediction in the Control of Reaching , 1999, Neural Computation.

[7]  Alessandra Pedrocchi,et al.  Distributed cerebellar plasticity implements generalized multiple-scale memory components in real-robot sensorimotor tasks , 2015, Front. Comput. Neurosci..

[8]  François Chapeau-Blondeau,et al.  A neural network model of the cerebellar cortex performing dynamic associations , 1991, Biological Cybernetics.

[9]  Yukie Nagai,et al.  Staged Development of Robot Skills: Behavior Formation, Affordance Learning and Imitation with Motionese , 2015, IEEE Transactions on Autonomous Mental Development.

[10]  Kevin C. Craig,et al.  Mechatronic design of a ball-on-plate balancing system , 2002 .

[11]  Eduardo Ros,et al.  Modeling the Cerebellar Microcircuit: New Strategies for a Long-Standing Issue , 2016, Front. Cell. Neurosci..

[12]  Joachim Hermsdörfer,et al.  The role of the cerebellum for predictive control of grasping , 2008, The Cerebellum.

[13]  Chih-Min Lin,et al.  Robust Fault-Tolerant Control for a Biped Robot Using a Recurrent Cerebellar Model Articulation Controller , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[14]  Snehasis Mukhopadhyay,et al.  Associative learning in random environments using neural networks , 1991, IEEE Trans. Neural Networks.

[15]  D. Wolpert,et al.  Internal models in the cerebellum , 1998, Trends in Cognitive Sciences.

[16]  J. Kocijan,et al.  Gaussian process model based predictive control , 2004, Proceedings of the 2004 American Control Conference.

[17]  Eduardo Ros,et al.  Fast convergence of learning requires plasticity between inferior olive and deep cerebellar nuclei in a manipulation task: a closed-loop robotic simulation , 2014, Front. Comput. Neurosci..

[18]  Wei He,et al.  Adaptive Neural Network Control of an Uncertain Robot With Full-State Constraints , 2016, IEEE Transactions on Cybernetics.

[19]  Tanaka.  The role of , 2000, Journal of insect physiology.

[20]  Jan Peters,et al.  Model Learning with Local Gaussian Process Regression , 2009, Adv. Robotics.

[21]  Wisama Khalil,et al.  Modeling, Identification and Control of Robots , 2003 .

[22]  Eduardo Ros,et al.  Distributed Cerebellar Motor Learning: A Spike-Timing-Dependent Plasticity Model , 2016, Front. Comput. Neurosci..

[23]  Henry Markram,et al.  Liquid Computing in a Simplified Model of Cortical Layer IV: Learning to Balance a Ball , 2012, ICANN.

[24]  S. Scott,et al.  Feedback control during voluntary motor actions , 2015, Current Opinion in Neurobiology.

[25]  伊藤 正男.  The cerebellum and neural control , 1984 .

[26]  Andreea C. Bostan,et al.  Consensus Paper: Towards a Systems-Level View of Cerebellar Function: the Interplay Between Cerebellum, Basal Ganglia, and Cortex , 2016, The Cerebellum.

[27]  Björn Wittenmark,et al.  Adaptive Dual Control Methods: An Overview , 1995 .

[28]  Mitsuo Kawato,et al.  Feedback-Error-Learning Neural Network for Supervised Motor Learning , 1990 .

[29]  Stephen E. Levinson,et al.  Vision Based Balancing Tasks for the iCub Platform : A Case Study for Learning External Dynamics , 2010 .

[30]  Juyang Weng,et al.  From neural networks to the brain: autonomous mental development , 2006, IEEE Computational Intelligence Magazine.

[31]  M. Kawato,et al.  A hierarchical neural-network model for control and learning of voluntary movement , 2004, Biological Cybernetics.

[32]  Shinya Aoi,et al.  Adaptation mechanism of interlimb coordination in human split-belt treadmill walking through learning of foot contact timing: a robotics study , 2015, Journal of The Royal Society Interface.

[33]  Jun Nakanishi,et al.  A Bayesian Approach to Nonlinear Parameter Identification for Rigid Body Dynamics , 2006, Robotics: Science and Systems.

[34]  James S. Albus,et al.  Theoretical and Experimental Aspects of a Cerebellar Model , 1973 .

[35]  Laurentiu S. Popa,et al.  What Features of Limb Movements are Encoded in the Discharge of Cerebellar Neurons? , 2011, The Cerebellum.

[36]  Masao Ito,et al.  Consensus Paper: The Cerebellum's Role in Movement and Cognition , 2013, The Cerebellum.

[37]  Shuzhi Sam Ge,et al.  Adaptive neural network control for strict-feedback nonlinear systems using backstepping design , 1999, Proceedings of the 1999 American Control Conference (Cat. No. 99CH36251).

[38]  Snehasis Mukhopadhyay,et al.  Adaptive control using neural networks and approximate models , 1997, IEEE Trans. Neural Networks.

[39]  Eduardo Ros,et al.  Cerebellarlike Corrective Model Inference Engine for Manipulation Tasks , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[40]  Kumpati S. Narendra,et al.  Gradient methods for the optimization of dynamical systems containing neural networks , 1991, IEEE Trans. Neural Networks.

[41]  Stephen H Scott,et al.  Context-dependent inhibition of unloaded muscles during the long-latency epoch. , 2015, Journal of neurophysiology.

[42]  Christopher G. Atkeson,et al.  Estimation of Inertial Parameters of Manipulator Loads and Links , 1986 .

[43]  Jun Nakanishi,et al.  Operational Space Control: A Theoretical and Empirical Comparison , 2008, Int. J. Robotics Res..

[44]  Chih-Min Lin,et al.  Robust Adaptive Tracking Control for Manipulators Based on a TSK Fuzzy Cerebellar Model Articulation Controller , 2018, IEEE Access.

[45]  Tadashi Yamazaki,et al.  The cerebellum as a liquid state machine , 2007, Neural Networks.

[46]  S P Wise,et al.  Distributed modular architectures linking basal ganglia, cerebellum, and cerebral cortex: their role in planning and controlling action. , 1995, Cerebral cortex.

[47]  Alessandra Pedrocchi,et al.  Model-Driven Analysis of Eyeblink Classical Conditioning Reveals the Underlying Structure of Cerebellar Plasticity and Neuronal Activity , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[48]  W. T. Miller,et al.  An overview of the CMAC neural network , 1991, [1991 Proceedings] IEEE Conference on Neural Networks for Ocean Engineering.

[49]  Egidio D'Angelo,et al.  Distributed cerebellar plasticity implements adaptable gain control in a manipulation task: a closed-loop robotic simulation , 2013, Front. Neural Circuits.

[50]  Mitsuo Kawato,et al.  Internal models for motor control and trajectory planning , 1999, Current Opinion in Neurobiology.

[51]  Stefan Schaal,et al.  Locally Weighted Projection Regression : An O(n) Algorithm for Incremental Real Time Learning in High Dimensional Space , 2000 .

[52]  D. Marr.  A theory of cerebellar cortex , 1969, The Journal of physiology.

[53]  W. T. Miller,et al.  CMAC: an associative neural network alternative to backpropagation , 1990, Proc. IEEE.

[54]  Yian Deng,et al.  How Does a Robot Develop Its Reaching Ability Like Human Infants Do? , 2018, IEEE Transactions on Cognitive and Developmental Systems.

[55]  Chun-Yi Su,et al.  Adaptive Neural Network Control for Robotic Manipulators With Unknown Deadzone , 2018, IEEE Transactions on Cybernetics.

[56]  Jun Nakanishi,et al.  Composite adaptive control with locally weighted statistical learning , 2005, Neural Networks.

[57]  Mitsuo Kawato,et al.  Feedback-error-learning neural network for trajectory control of a robotic manipulator , 1988, Neural Networks.

[58]  Giulio Sandini,et al.  A Survey of Artificial Cognitive Systems: Implications for the Autonomous Development of Mental Capabilities in Computational Agents , 2007, IEEE Transactions on Evolutionary Computation.

[59]  Paolo Dario,et al.  Adaptive gaze stabilization through cerebellar internal models in a humanoid robot , 2016, 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[60]  Juyang Weng,et al.  Autonomous Mental Development , 2011, Intelligent Systems.

[61]  M. Kawato,et al.  New insights into olivo-cerebellar circuits for learning from a small training sample , 2017, Current Opinion in Neurobiology.

[62]  Dean V. Buonomano,et al.  Neural Network Model of the Cerebellum: Temporal Discrimination and the Timing of Motor Responses , 1999, Neural Computation.

[63]  Rüdiger Dillmann,et al.  Connecting Artificial Brains to Robots in a Comprehensive Simulation Framework: The Neurorobotics Platform , 2017, Front. Neurorobot..

[64]  K. Narendra,et al.  Persistent excitation in adaptive systems , 1987 .

[65]  Anuradha M. Annaswamy,et al.  Adaptive control of simple time-varying systems , 1989, Proceedings of the 28th IEEE Conference on Decision and Control,.

[66]  Chun-Sheng Chen.  Sliding-mode-based fuzzy CMAC controller design for a class of uncertain nonlinear system , 2009, 2009 IEEE International Conference on Systems, Man and Cybernetics.

[67]  M. Fujita,et al.  Simulation of adaptive modification of the vestibulo-ocular reflex with an adaptive filter model of the cerebellum , 1982, Biological Cybernetics.

[68]  John Porrill,et al.  Recurrent Cerebellar Loops Simplify Adaptive Control of Redundant and Nonlinear Motor Systems , 2007, Neural Computation.

[69]  N. Donegan,et al.  A model of Pavlovian eyelid conditioning based on the synaptic organization of the cerebellum. , 1997, Learning & memory.

[70]  Eduardo Ros,et al.  Adaptive Robotic Control Driven by a Versatile Spiking Cerebellar Network , 2014, PloS one.

[71]  W. Thomas Miller,et al.  Sensor-based control of robotic manipulators using a general learning algorithm , 1987, IEEE J. Robotics Autom..

[72]  Juyang Weng,et al.  Developmental Robots: Theory, Method and Experimental Results , 1999 .

[73]  M. Ito.  Cerebellar microcomplexes. , 1997, International review of neurobiology.

[74]  J. Albus.  A Theory of Cerebellar Function , 1971 .

[75]  Snehasis Mukhopadhyay,et al.  Multilevel control of a dynamical systems using neural networks , 1991, [1991] Proceedings of the 30th IEEE Conference on Decision and Control.

[76]  Giulio Sandini,et al.  A developmental approach to visually-guided reaching in artificial systems , 1999, Neural Networks.

[77]  H. Unbehauen,et al.  Identification of nonlinear systems , 2019, 2019 Proceedings of the Conference on Control and its Applications.

[78]  Jan Peters,et al.  Model learning for robot control: a survey , 2011, Cognitive Processing.

[79]  Silvia Tolu,et al.  A Scalable Neuro-inspired Robot Controller Integrating a Machine Learning Algorithm and a Spiking Cerebellar-Like Network , 2017, Living Machines.

[80]  Masao Ito.  Control of mental activities by internal models in the cerebellum , 2008, Nature Reviews Neuroscience.

[81]  Ricardo O. Carelli,et al.  Neural networks for advanced control of robot manipulators , 2002, IEEE Trans. Neural Networks.

[82]  Andrew Howard,et al.  Design and use paradigms for Gazebo, an open-source multi-robot simulator , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[83]  J. Schmahmann,et al.  Disorders of the cerebellum: ataxia, dysmetria of thought, and the cerebellar cognitive affective syndrome. , 2004, The Journal of neuropsychiatry and clinical neurosciences.

[84]  Silvia Tolu,et al.  Adaptive and Predictive Control of a Simulated Robot arm , 2013, Int. J. Neural Syst..

[85]  P. Dean,et al.  The cerebellar microcircuit as an adaptive filter: experimental and computational evidence , 2010, Nature Reviews Neuroscience.

[86]  Eduardo D. Sontag,et al.  Feedback Stabilization Using Two-Hidden-Layer Nets , 1991, 1991 American Control Conference.

[87]  Silvia Tolu,et al.  Bio-inspired adaptive feedback error learning architecture for motor control , 2012, Biological Cybernetics.