A Brief Review of Neural Networks Based Learning and Control and Their Applications for Robots

As an imitation of the biological nervous systems, neural networks (NN), which are characterized with powerful learning ability, have been employed in a wide range of applications, such as control of complex nonlinear systems, optimization, system identification and patterns recognition etc. This article aims to bring a brief review of the state-of-art NN for the complex nonlinear systems. Recent progresses of NNs in both theoretical developments and practical applications are investigated and surveyed. Specifically, NN based robot learning and control applications were further reviewed, including NN based robot manipulator control, NN based human robot interaction and NN based behavior recognition and generation.

[1]  Robert M. Sanner,et al.  Gaussian Networks for Direct Adaptive Control , 1991, 1991 American Control Conference.

[2]  Long Cheng,et al.  Tracking Control of a Closed-Chain Five-Bar Robot With Two Degrees of Freedom by Integration of an Approximation-Based Approach and Mechanical Design , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[3]  Rajesh Kumar,et al.  Energy analysis of a building using artificial neural network: A review , 2013 .

[4]  Angelo Cangelosi,et al.  Toward abstraction from multi-modal data: Empirical studies on multiple time-scale recurrent models , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[5]  Angelo Cangelosi,et al.  Toward a self-organizing pre-symbolic neural model representing sensorimotor primitives , 2014, Front. Behav. Neurosci..

[6]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[7]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[8]  Jing Na,et al.  Global adaptive tracking control of robot manipulators using neural networks with finite-time learning convergence , 2017 .

[9]  Frank L. Lewis,et al.  A dynamic recurrent neural-network-based adaptive observer for a class of nonlinear systems , 1997, Autom..

[10]  Chaoxu Mu,et al.  Neural-network-based adaptive guaranteed cost control of nonlinear dynamical systems with matched uncertainties , 2017, Neurocomputing.

[11]  Zhi Liu,et al.  Personalized Variable Gain Control With Tremor Attenuation for Robot Teleoperation , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[12]  Qinglai Wei,et al.  Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming , 2012, Autom..

[13]  Jing Li,et al.  Trajectory Planning and Optimized Adaptive Control for a Class of Wheeled Inverted Pendulum Vehicle Models , 2013, IEEE Transactions on Cybernetics.

[14]  Frank L. Lewis,et al.  Neural-network approximation of piecewise continuous functions: application to friction compensation , 2002, IEEE Trans. Neural Networks.

[15]  Stefan Wermter,et al.  Robot Trajectory Prediction and Recognition Based on a Computational Mirror Neurons Model , 2011, ICANN.

[16]  Frank L. Lewis,et al.  Adaptive cooperative tracking control of higher-order nonlinear systems with unknown dynamics , 2012, Autom..

[17]  James S. Albus,et al.  Theoretical and Experimental Aspects of a Cerebellar Model , 1973 .

[18]  Frank L. Lewis,et al.  Neural Network Control of Robot Manipulators , 1996, IEEE Expert.

[19]  Luca Maria Gambardella,et al.  Deep Big Simple Neural Nets Excel on Handwritten Digit Recognition , 2010, ArXiv.

[20]  José de Jesús Rubio,et al.  USNFIS: Uniform stable neuro fuzzy inference system , 2017, Neurocomputing.

[21]  Yun Li,et al.  Artificial evolution of neural networks and its application to feedback control , 1996, Artif. Intell. Eng..

[22]  Shuzhi Sam Ge,et al.  Output feedback adaptive control of a class of nonlinear discrete-time systems with unknown control directions , 2009, Autom..

[23]  Keng Peng Tee,et al.  A Framework of Human–Robot Coordination Based on Game Theory and Policy Iteration , 2016, IEEE Transactions on Robotics.

[24]  Shuzhi Sam Ge,et al.  Impedance Learning for Robots Interacting With Unknown Environments , 2014, IEEE Transactions on Control Systems Technology.

[25]  Meng Joo Er,et al.  Adaptive fuzzy PD control with stable H∞ tracking guarantee , 2017, Neurocomputing.

[26]  Yanan Li,et al.  Haptic Identification by ELM-Controlled Uncertain Manipulator , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[27]  Jun Tani,et al.  Development of compositional and contextual communicable congruence in robots by using dynamic neural network models , 2015, Neural Networks.

[28]  Frank L. Lewis,et al.  Multilayer discrete-time neural-net controller with guaranteed performance , 1996, IEEE Trans. Neural Networks.

[29]  Hongliang Guo,et al.  A type of biased consensus-based distributed neural network for path planning , 2017 .

[30]  F. Lewis,et al.  Reinforcement Learning and Feedback Control: Using Natural Decision Methods to Design Optimal Adaptive Controllers , 2012, IEEE Control Systems.

[31]  Frank L. Lewis,et al.  CMAC neural network control of robot manipulators , 1997 .

[32]  Qingquan Li,et al.  Reinforcement learning control for coordinated manipulation of multi-robots , 2015, Neurocomputing.

[33]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[34]  Jun Tani,et al.  Self-organization of distributedly represented multiple behavior schemata in a mirror system: reviews of robot experiments using RNNPB , 2004, Neural Networks.

[35]  Larry D. Pyeatt,et al.  A comparison between cellular encoding and direct encoding for genetic neural networks , 1996 .

[36]  Iztok Fister,et al.  Artificial neural network regression as a local search heuristic for ensemble strategies in differential evolution , 2015, Nonlinear Dynamics.

[37]  Lingfeng Wang,et al.  Book Review Evolutionary Robotics: from Algorithms to Implementations Evolutionary Robotics: from Algorithms to Implementations , 2022 .

[38]  Shuzhi Sam Ge,et al.  Output Feedback NN Control for Two Classes of Discrete-Time Systems With Unknown Control Directions in a Unified Approach , 2008, IEEE Transactions on Neural Networks.

[39]  Jing Na.  Time-delay Positive Feedback Control for Nonlinear Time-delay Systems with Neural Network Compensation: Time-delay Positive Feedback Control for Nonlinear Time-delay Systems with Neural Network Compensation , 2009 .

[40]  Kay Chen Tan,et al.  Grey-box model identification via evolutionary computing , 2002 .

[41]  Shuzhi Sam Ge,et al.  Optimal Critic Learning for Robot Control in Time-Varying Environments , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[42]  J. Tani.  Exploring Robotic Minds: Actions, Symbols, and Consciousness as Self-Organizing Dynamic Phenomena , 2016 .

[43]  Luba Zuk,et al.  Fetal and infant spontaneous general movements as predictors of developmental disabilities. , 2011, Developmental disabilities research reviews.

[44]  Wei He,et al.  Adaptive Neural Network Control of a Marine Vessel With Constraints Using the Asymmetric Barrier Lyapunov Function. , 2017, IEEE transactions on cybernetics.

[45]  F. L. Lewis,et al.  Neural-network predictive control for nonlinear dynamic systems with time-delay , 2003, IEEE Trans. Neural Networks.

[46]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[47]  Shuzhi Sam Ge,et al.  Adaptive neural network control for strict-feedback nonlinear systems using backstepping design , 1999, Proceedings of the 1999 American Control Conference (Cat. No. 99CH36251).

[48]  Yangmin Li,et al.  H∞ synchronization of coupled reaction-diffusion neural networks with mixed delays , 2016, Complex..

[49]  Huaguang Zhang,et al.  Neural-Network-Based Near-Optimal Control for a Class of Discrete-Time Affine Nonlinear Systems With Control Constraints , 2009, IEEE Transactions on Neural Networks.

[50]  Jin Zhang,et al.  Neural-network control of nonaffine nonlinear system with zero dynamics by state and output feedback , 2003, IEEE Trans. Neural Networks.

[51]  Changyin Sun,et al.  Neural Network Control of a Robotic Manipulator With Input Deadzone and Output Constraint , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[52]  Xin Yao,et al.  A new evolutionary system for evolving artificial neural networks , 1997, IEEE Trans. Neural Networks.

[53]  Paul J. Werbos,et al.  Approximate dynamic programming for real-time control and neural modeling , 1992 .

[54]  Keng Peng Tee,et al.  Continuous Role Adaptation for Human–Robot Shared Control , 2015, IEEE Transactions on Robotics.

[55]  Wilfrid Perruquetti,et al.  Experimental tests of a sliding mode controller for trajectory tracking of a car-like mobile robot , 2014, Robotica.

[56]  Luca Maria Gambardella,et al.  Deep, Big, Simple Neural Nets for Handwritten Digit Recognition , 2010, Neural Computation.

[57]  Frank L. Lewis,et al.  Identification of nonlinear dynamical systems using multilayered neural networks , 1996, Autom..

[58]  Frank L. Lewis,et al.  2009 Special Issue: Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems , 2009 .

[59]  Stefan Wermter,et al.  Learning Features and Predictive Transformation Encoding Based on a Horizontal Product Model , 2012, ICANN.

[60]  Guang Li,et al.  Adaptive Neural Network Feedforward Control for Dynamically Substructured Systems , 2014, IEEE Transactions on Control Systems Technology.

[61]  Shuzhi Sam Ge,et al.  Adaptive Neural Output Feedback Control of Uncertain Nonlinear Systems With Unknown Hysteresis Using Disturbance Observer , 2015, IEEE Transactions on Industrial Electronics.

[62]  Chenguang Yang,et al.  Global Neural Dynamic Surface Tracking Control of Strict-Feedback Systems With Application to Hypersonic Flight Vehicle , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[63]  Jun Yu,et al.  Machine learning and signal processing for big multimedia analysis , 2017, Neurocomputing.

[64]  Chenguang Yang,et al.  Neural-Learning-Based Telerobot Control With Guaranteed Performance , 2017, IEEE Transactions on Cybernetics.

[65]  Jing Na,et al.  Adaptive Control for Nonlinear Pure-Feedback Systems With High-Order Sliding Mode Observer , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[66]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[67]  K. Tan.  GREY-BOX MODEL IDENTIFICATION VIA EVOLUTIONARY COMPUTING , 2007 .

[68]  Shuzhi Sam Ge,et al.  Neural networks impedance control of robots interacting with environments , 2013 .

[69]  Weisheng Yan,et al.  Mutual Synchronization of Multiple Robot Manipulators with Unknown Dynamics , 2012, J. Intell. Robotic Syst..

[70]  W. Prinz.  Perception and Action Planning , 1997 .

[71]  Yu Guo,et al.  Adaptive neural network predictive control for nonlinear pure feedback systems with input delay , 2012 .

[72]  Jun Tani,et al.  Self-organization of distributedly represented multiple behavior schemata in a mirror system: reviews of robot experiments using RNNPB [Neural Networks 17 (8–9) 1273–1289] , 2005 .

[73]  Tarun Khanna,et al.  Foundations of neural networks , 1990 .

[74]  José de Jesús Rubio,et al.  Uniform stable radial basis function neural network for the prediction in two mechatronic processes , 2017, Neurocomputing.

[75]  J. Na,et al.  Online adaptive approximate optimal tracking control with simplified dual approximation structure for continuous-time unknown nonlinear systems , 2014, IEEE/CAA Journal of Automatica Sinica.

[76]  Hava T. Siegelmann,et al.  On the Computational Power of Neural Nets , 1995, J. Comput. Syst. Sci..

[77]  Yang Li,et al.  Adaptive Neural Network Control of AUVs With Control Input Nonlinearities Using Reinforcement Learning , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[78]  Angelo Cangelosi,et al.  Sensorimotor input as a language generalisation tool: a neurorobotics model for generation and generalisation of noun-verb combinations with sensorimotor inputs , 2016, Autonomous Robots.

[79]  Jing Na,et al.  Adaptive neural dynamic surface control for servo systems with unknown dead-zone , 2011 .

[80]  Bo Yang,et al.  Robust Hybrid Control Based on PD and Novel CMAC With Improved Architecture and Learning Scheme for Electric Load Simulator , 2014, IEEE Transactions on Industrial Electronics.

[81]  Beibei Ren,et al.  Deterministic learning enhanced neutral network control of unmanned helicopter , 2016 .

[82]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[83]  Zhongke Shi,et al.  Composite Neural Dynamic Surface Control of a Class of Uncertain Nonlinear Systems in Strict-Feedback Form , 2014, IEEE Transactions on Cybernetics.

[84]  Huaguang Zhang,et al.  Adaptive Dynamic Programming: An Introduction , 2009, IEEE Computational Intelligence Magazine.

[85]  Shuzhi Sam Ge,et al.  Adaptive neural control of nonlinear time-delay systems with unknown virtual control coefficients , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[86]  Tetsuya Ogata,et al.  Emergence of hierarchical structure mirroring linguistic composition in a recurrent neural network , 2011, Neural Networks.

[87]  José Jesús Rubio.  Adaptive least square control in discrete time of robotic arms , 2015 .

[88]  Lixin Han,et al.  Finite-time stochastic synchronization of time-delay neural networks with noise disturbance , 2017 .

[89]  Gang Tao,et al.  Dynamic Surface Control Using Neural Networks for a Class of Uncertain Nonlinear Systems With Input Saturation , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[90]  Yurong Liu,et al.  A survey of deep neural network architectures and their applications , 2017, Neurocomputing.

[91]  Jun Tang,et al.  Formation of Autapse Connected to Neuron and Its Biological Function , 2017, Complex..

[92]  Xin Liu,et al.  Neuro-adaptive observer based control of flexible joint robot , 2018, Neurocomputing.

[93]  Jing Wang,et al.  Robust adaptive neural control for a class of perturbed strict feedback nonlinear systems , 2002, IEEE Trans. Neural Networks.

[94]  Long Cheng,et al.  Adaptive neural network tracking control for manipulators with uncertain kinematics, dynamics and actuator model , 2009, Autom..

[95]  Junpei Zhong,et al.  Artificial Neural Models for Feedback Pathways for Sensorimotor Integration , 2015 .

[96]  Chenguang Yang,et al.  Neural Network-Based Motion Control of an Underactuated Wheeled Inverted Pendulum Model , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[97]  Changyin Sun,et al.  Adaptive Neural Impedance Control of a Robotic Manipulator With Input Saturation , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[98]  Lola Cañamero,et al.  From continuous affective space to continuous expression space: Non-verbal behaviour recognition and generation , 2014, 4th International Conference on Development and Learning and on Epigenetic Robotics.

[99]  Stefan Wermter,et al.  A predictive network architecture for a robust and smooth robot docking behavior , 2012, Paladyn J. Behav. Robotics.

[100]  Michael Defoort,et al.  Sliding-Mode Formation Control for Cooperative Autonomous Mobile Robots , 2008, IEEE Transactions on Industrial Electronics.

[101]  Jing Na,et al.  Time-delay Positive Feedback Control for Nonlinear Time-delay Systems with Neural Network Compensation , 2008 .

[102]  Kay Chen Tan,et al.  Hybrid Multiobjective Evolutionary Design for Artificial Neural Networks , 2008, IEEE Transactions on Neural Networks.

[103]  Shuzhi Sam Ge,et al.  Direct Adaptive Neural Control for a Class of Uncertain Nonaffine Nonlinear Systems Based on Disturbance Observer , 2013, IEEE Transactions on Cybernetics.

[104]  José de Jesús Rubio,et al.  Adaptive least square control in discrete time of robotic arms , 2015, Soft Comput..

[105]  Victor C. M. Leung,et al.  Applying a new localized generalization error model to design neural networks trained with extreme learning machine , 2014, Neural Computing and Applications.

[106]  Fuad E. Alsaadi,et al.  Global lagrange stability of complex-valued neural networks of neutral type with time-varying delays , 2016, Complex..

[107]  Zhongke Shi,et al.  DOB-Based Neural Control of Flexible Hypersonic Flight Vehicle Considering Wind Effects , 2017, IEEE Transactions on Industrial Electronics.

[108]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[109]  James S. Albus,et al.  New Approach to Manipulator Control: The Cerebellar Model Articulation Controller (CMAC)1 , 1975 .

[110]  Hamed K. Abbas,et al.  Modelling the colonisation of maize by toxigenic and non-toxigenic Aspergillus flavus strains: implications for biological control , 2008 .

[111]  E. Michael Azoff,et al.  Neural Network Time Series: Forecasting of Financial Markets , 1994 .

[112]  Changyin Sun,et al.  Neural Network Control of a Flexible Robotic Manipulator Using the Lumped Spring-Mass Model , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[113]  Haibo He,et al.  Adaptive Learning and Control for MIMO System Based on Adaptive Dynamic Programming , 2011, IEEE Transactions on Neural Networks.

[114]  Shuzhi Sam Ge,et al.  Adaptive Predictive Control Using Neural Network for a Class of Pure-Feedback Systems in Discrete Time , 2008, IEEE Transactions on Neural Networks.

[115]  Chenguang Yang,et al.  A Review of Fuzzy Logic and Neural Network Based Intelligent Control Design for Discrete-Time Systems , 2016 .

[116]  José de Jesús Rubio.  Interpolation neural network model of a manufactured wind turbine , 2015, Neural Computing and Applications.

[117]  Xin Zhang,et al.  Adaptive sliding-mode attitude control for autonomous underwater vehicles with input nonlinearities , 2016 .

[118]  Yun Zhang,et al.  Adaptive robust fuzzy control for dual arm robot with unknown input deadzone nonlinearity , 2015, Nonlinear Dynamics.

[119]  Hugo de Garis,et al.  An artificial brain ATR's CAM-Brain Project aims to build/evolve an artificial brain with a million neural net modules inside a trillion cell Cellular Automata Machine , 1994, New Generation Computing.

[120]  Zhitao Lin,et al.  Relative ordering learning in spiking neural network for pattern recognition , 2018, Neurocomputing.

[121]  M. Nour.  Surfing Uncertainty: Prediction, Action, and the Embodied Mind. , 2017, British Journal of Psychiatry.

[122]  Wei He,et al.  Adaptive Neural Network Control of an Uncertain Robot With Full-State Constraints , 2016, IEEE Transactions on Cybernetics.

[123]  Frank L. Lewis,et al.  Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[124]  Jun Tani,et al.  How can a recurrent neurodynamic predictive coding model cope with fluctuation in temporal patterns? Robotic experiments on imitative interaction , 2017, Neural Networks.

[125]  Chun-Yi Su,et al.  Neural Control of Bimanual Robots With Guaranteed Global Stability and Motion Precision , 2017, IEEE Transactions on Industrial Informatics.

[126]  Derong Liu,et al.  Policy Iteration Adaptive Dynamic Programming Algorithm for Discrete-Time Nonlinear Systems , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[127]  José de Jesús Rubio,et al.  Discrete time control based in neural networks for pendulums , 2017, Appl. Soft Comput..

[128]  Shuzhi Sam Ge,et al.  Human–Robot Collaboration Based on Motion Intention Estimation , 2014, IEEE/ASME Transactions on Mechatronics.

[129]  Vincenzo Piuri,et al.  A neural-network based control solution to air-fuel ratio control for automotive fuel-injection systems , 2003, IEEE Trans. Syst. Man Cybern. Part C.

[130]  Hui Li,et al.  Evolutionary artificial neural networks: a review , 2011, Artificial Intelligence Review.

[131]  Wei He,et al.  Vibration Control of a Flexible Robotic Manipulator in the Presence of Input Deadzone , 2017, IEEE Transactions on Industrial Informatics.

[132]  F.L. Lewis,et al.  Reinforcement learning and adaptive dynamic programming for feedback control , 2009, IEEE Circuits and Systems Magazine.

[133]  Jun Tani,et al.  Bridging the Gap Between Probabilistic and Deterministic Models: A Simulation Study on a Variational Bayes Predictive Coding Recurrent Neural Network Model , 2017, ICONIP.

[134]  Derong Liu,et al.  Reinforcement-Learning-Based Robust Controller Design for Continuous-Time Uncertain Nonlinear Systems Subject to Input Constraints , 2015, IEEE Transactions on Cybernetics.

[135]  Shuzhi Sam Ge,et al.  Learning impedance control for physical robot–environment interaction , 2012, Int. J. Control.

[136]  Xingjian Wang,et al.  Teleoperation Control Based on Combination of Wave Variable and Neural Networks , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[137]  Frank L. Lewis,et al.  CMAC neural networks for control of nonlinear dynamical systems: Structure, stability and passivity , 1997, Autom..

[138]  Yu Guo,et al.  Online adaptive optimal control for continuous-time nonlinear systems with completely unknown dynamics , 2016, Int. J. Control.

[139]  Haibo He,et al.  A three-network architecture for on-line learning and optimization based on adaptive dynamic programming , 2012, Neurocomputing.

[140]  Yu Guo,et al.  Adaptive Prescribed Performance Motion Control of Servo Mechanisms with Friction Compensation , 2014, IEEE Transactions on Industrial Electronics.

[141]  Jun Zhang,et al.  Evolutionary Computation Meets Machine Learning: A Survey , 2011, IEEE Computational Intelligence Magazine.

[142]  Bryan C. Pijanowski,et al.  A big data urban growth simulation at a national scale: Configuring the GIS and neural network based Land Transformation Model to run in a High Performance Computing (HPC) environment , 2014, Environ. Model. Softw..