A recurrent neural network based MPC for a hybrid neuroprosthesis system

Control input sequence in a hybrid neuroprosthesis that combines functional electrical stimulation (FES) and an electric motor can be optimized by a model based optimization method, like model predictive control (MPC). However, because the human muscle model is highly nonlinear, time-varying, and contains unmeasurable state variables, it is often difficult to identify the model. Therefore, a three-layer recurrence neural network (RNN) is developed in this paper, in which backpropagation through time (BPTT) is used as training technique and the internal states are used to represent the unmeasurable states. This structure shows the potential to approximate the model of the hybrid neuroprosthesis system. After the NN model is obtained, an adaptive model predictive control is used to simulate regulation and tracking tasks to test the performance of the NN training and the MPC method.

[1]  Naji A. Alibeji,et al.  Model-Based Dynamic Control Allocation in a Hybrid Neuroprosthesis , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[2]  Steffen Udluft,et al.  Solving Partially Observable Reinforcement Learning Problems with Recurrent Neural Networks , 2012, Neural Networks: Tricks of the Trade.

[3]  Vincent A Akpan,et al.  Nonlinear model identification and adaptive model predictive control using neural networks. , 2011, ISA transactions.

[4]  Nitin Sharma,et al.  A semi-active hybrid neuroprosthesis for restoring lower limb function in paraplegics , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[5]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[6]  J.J. Steil,et al.  Backpropagation-decorrelation: online recurrent learning with O(N) complexity , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[7]  Knut Graichen,et al.  A Real-Time Gradient Method for Nonlinear Model Predictive Control , 2012 .

[8]  Nitin Sharma,et al.  Dynamic control allocation of a feedback linearized hybrid neuroprosthetic system , 2016, 2016 American Control Conference (ACC).

[9]  U. R. Prasad,et al.  Back propagation through adjoints for the identification of nonlinear dynamic systems using recurrent neural models , 1994, IEEE Trans. Neural Networks.

[10]  R. H. White.  The learning rate in back-propagation systems: an application of Newton's method , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[11]  R. J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[12]  E. Marsolais,et al.  Synthesis of paraplegic gait with multichannel functional neuromuscular stimulation , 1994 .

[13]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[14]  Eric A. Wan,et al.  Relating Real-Time Backpropagation and Backpropagation-Through-Time: An Application of Flow Graph Interreciprocity , 1994, Neural Computation.

[15]  Frank L. Lewis,et al.  Robot Manipulator Control: Theory and Practice , 2003 .

[16]  Léon Bottou,et al.  The Tradeoffs of Large Scale Learning , 2007, NIPS.

[17]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[18]  Hui Li,et al.  A Stochastic-Based FPGA Controller for an Induction Motor Drive With Integrated Neural Network Algorithms , 2008, IEEE Transactions on Industrial Electronics.

[19]  William K. Durfee.  Gait Restoration by Functional Electrical Stimulation , 2005, CLAWAR.

[20]  Steffen Udluft,et al.  A Neural Reinforcement Learning Approach to Gas Turbine Control , 2007, 2007 International Joint Conference on Neural Networks.

[21]  C. L. Lynch,et al.  A Comparison of Closed-Loop Control Algorithms for Regulating Electrically Stimulated Knee Movements in Individuals With Spinal Cord Injury , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[22]  Seul Jung,et al.  Hardware Implementation of a Real-Time Neural Network Controller With a DSP and an FPGA for Nonlinear Systems , 2007, IEEE Transactions on Industrial Electronics.

[23]  Naji A. Alibeji,et al.  Nonlinear model predictive control of functional electrical stimulation , 2017 .

[24]  Michael Goldfarb,et al.  An Approach for the Cooperative Control of FES With a Powered Exoskeleton During Level Walking for Persons With Paraplegia , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[25]  David Q. Mayne,et al.  Constrained model predictive control: Stability and optimality , 2000, Autom..

[26]  Danilo P. Mandic,et al.  Relating the Slope of the Activation Function and the Learning Rate Within a Recurrent Neural Network , 1999, Neural Computation.

[27]  K S Narendra,et al.  Control of nonlinear dynamical systems using neural networks. II. Observability, identification, and control , 1996, IEEE Trans. Neural Networks.

[28]  K. Graichen.  0 A Real-Time Gradient Method for Nonlinear Model Predictive Control , 2012 .

[29]  Michael I. Jordan,et al.  Forward Models: Supervised Learning with a Distal Teacher , 1992, Cogn. Sci..

[30]  S. Udluft,et al.  A Recurrent Control Neural Network for Data Efficient Reinforcement Learning , 2007, 2007 IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning.

[31]  Yishay Mansour,et al.  Policy Gradient Methods for Reinforcement Learning with Function Approximation , 1999, NIPS.

[32]  P J Webros.  BACKPROPAGATION THROUGH TIME: WHAT IT DOES AND HOW TO DO IT , 1990 .

[33]  Steffen Udluft,et al.  The Recurrent Control Neural Network , 2007, ESANN.

[34]  Hans-Georg Zimmermann,et al.  Recurrent Neural Networks are Universal approximators , 2007, Int. J. Neural Syst..

[35]  R B Stein,et al.  Optimal control of walking with functional electrical stimulation: a computer simulation study. , 1999, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[36]  Naji A. Alibeji,et al.  An adaptive low-dimensional control to compensate for actuator redundancy and FES-induced muscle fatigue in a hybrid neuroprosthesis , 2017 .

[37]  Marcus Johnson,et al.  Closed-Loop Neural Network-Based NMES Control for Human Limb Tracking , 2012, IEEE Transactions on Control Systems Technology.