Online-Learning Deep Neuro-Adaptive Dynamic Inversion Controller for Model Free Control

Adaptive methods are popular within the control literature due to the flexibility and forgiveness they offer in the area of modelling. Neural network adaptive control is favorable specifically for the powerful nature of the machine learning algorithm to approximate unknown functions and for the ability to relax certain constraints within traditional adaptive control. Deep neural networks are large framework networks with vastly superior approximation characteristics than their shallow counterparts. However, implementing a deep neural network can be difficult due to size specific complications such as vanishing/exploding gradients in training. In this paper, a neuro-adaptive controller is implemented featuring a deep neural network trained on a new weight update law that escapes the vanishing/exploding gradient problem by only incorporating the sign of the gradient. The type of controller designed is an adaptive dynamic inversion controller utilizing a modified state observer in a secondary estimation loop to train the network. The deep neural network learns the entire plant model on-line, creating a controller that is completely model free. The controller design is tested in simulation on a 2 link planar robot arm. The controller is able to learn the nonlinear plant quickly and displays good performance in the tracking control problem.

[1]  Sergey Levine,et al.  Learning deep control policies for autonomous aerial vehicles with MPC-guided policy search , 2015, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[2]  Yisong Yue,et al.  On the distance between two neural networks and the stability of learning , 2020, NeurIPS.

[3]  Benjamin Karg,et al.  Efficient Representation and Approximation of Model Predictive Control Laws via Deep Learning , 2018, IEEE Transactions on Cybernetics.

[4]  Jun Fu,et al.  Missile Guidance Law Based on Robust Model Predictive Control Using Neural-Network Optimization , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[5]  Yisong Yue,et al.  Learning compositional functions via multiplicative weight updates , 2020, NeurIPS.

[6]  Tianshu Wei,et al.  Deep reinforcement learning: Framework, applications, and embedded implementations: Invited paper , 2017, 2017 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[7]  Radhakant Padhi,et al.  Model-following neuro-adaptive control design for non-square, non-affine nonlinear systems , 2007 .

[8]  Yan Wang,et al.  Unmanned Surface Vehicle Course Tracking Control Based on Neural Network and Deep Deterministic Policy Gradient Algorithm , 2018, 2018 OCEANS - MTS/IEEE Kobe Techno-Oceans (OTO).

[9]  Sanjeev Arora,et al.  The Multiplicative Weights Update Method: a Meta-Algorithm and Applications , 2012, Theory Comput..

[10]  Xiang Li,et al.  Attitude Estimation of Unmanned Aerial Vehicle Based on LSTM Neural Network , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[11]  Zhenghua Liu,et al.  Learning to Guide: Guidance Law Based on Deep Meta-Learning and Model Predictive Path Integral Control , 2019, IEEE Access.

[12]  James J. Q. Yu,et al.  Online Vehicle Routing With Neural Combinatorial Optimization and Deep Reinforcement Learning , 2019, IEEE Transactions on Intelligent Transportation Systems.

[13]  Nhan T. Nguyen,et al.  Neuroadaptive Model Following Controller Design for Non-affine Non-square Aircraft System , 2009 .

[14]  Yue Gao,et al.  Online Hybrid Learning to Speed Up Deep Reinforcement Learning Method for Commercial Aircraft Control , 2019, 2019 3rd International Symposium on Autonomous Systems (ISAS).

[15]  Huang Yi,et al.  A study on Deep Neural Networks framework , 2016, 2016 IEEE Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC).

[16]  S. Jagannathan,et al.  Event Triggered Neuro-Adaptive Controller (ETNAC) Design for Uncertain Linear Systems , 2018, 2018 IEEE Conference on Decision and Control (CDC).

[17]  Erdal Kayacan,et al.  Online Deep Fuzzy Learning for Control of Nonlinear Systems Using Expert Knowledge , 2020, IEEE Transactions on Fuzzy Systems.

[18]  Siddharth Krishna Kumar,et al.  On weight initialization in deep neural networks , 2017, ArXiv.

[19]  Yuanqing Xia,et al.  Six-DOF Spacecraft Optimal Trajectory Planning and Real-Time Attitude Control: A Deep Neural Network-Based Approach , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[20]  Erdal Kayacan,et al.  Online Deep Learning for Improved Trajectory Tracking of Unmanned Aerial Vehicles Using Expert Knowledge , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[21]  Xiufen Ye,et al.  Online Adaptive Critic Learning Control of Unknown Dynamics With Application to Deep Submergence Rescue Vehicle , 2020, IEEE Access.