Neural Network Fitting for Input-Output Manifolds in Constrained Linear Systems

This work presents a recent contribution regarding the application of multi-layer perceptron neural networks (MLP-NNs) to the fitting of complex piecewise affine control and observation laws in constrained linear systems. Such input-output maps arise from the imposition of the optimal contraction rate trajectory for the system state, or error in the observer case, within a given invariant polyhedral set that enforces the systems constraints, or in the case of optimal control laws in model predictive control (MPC). Although an offline law can be computed via multi-parametric optimization in the state feedback case, the number of regions that define the piecewise affine law can be very large, resulting in high hardware storage requirements and difficulty in quickly locating the state. On the other hand, online laws are usually more expensive in the runtime sense and can become infeasible for application in fast dynamics systems, unless an advanced, expensive processor is employed. From the data obtained by the simulation of online laws, MLPs can be trained to emulate such maps; thus, they can replace online computation and drastically reduce the runtime. Two numerical examples, one of which is based on a two-tank hydraulic system model, are presented to illustrate the proposed approach, with detailed design for constrained error estimation as well as static and dynamic output feedback.

[1]  Y. Candau,et al.  Set membership state and parameter estimation for systems described by nonlinear differential equations , 2004, Autom..

[2]  Manfred Morari,et al.  Robust constrained model predictive control using linear matrix inequalities , 1994, Proceedings of 1994 American Control Conference - ACC '94.

[3]  C. L. Philip Chen,et al.  An incremental adaptive implementation of functional-link processing for function approximation, time-series prediction, and system identification , 1998, Neurocomputing.

[4]  Concha Bielza,et al.  Machine Learning in Bioinformatics , 2008, Encyclopedia of Database Systems.

[5]  Jeff S. Shamma,et al.  Set-valued observers and optimal disturbance rejection , 1999, IEEE Trans. Autom. Control..

[6]  Pedro Ponce,et al.  The development of an artificial organic networks toolkit for LabVIEW , 2015, J. Comput. Chem..

[7]  Tomislav Lipic,et al.  Implementation framework for Artificial Neural Networks on FPGA , 2011, 2011 Proceedings of the 34th International Convention MIPRO.

[8]  Thomas Martinetz,et al.  'Neural-gas' network for vector quantization and its application to time-series prediction , 1993, IEEE Trans. Neural Networks.

[9]  Indranil Saha,et al.  journal homepage: www.elsevier.com/locate/neucom , 2022 .

[10]  Shlomo Geva,et al.  A constructive method for multivariate function approximation by multilayer perceptrons , 1992, IEEE Trans. Neural Networks.

[11]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[12]  Hannu T. Toivonen,et al.  Neural network approximation of a nonlinear model predictive controller applied to a pH neutralization process , 2005, Comput. Chem. Eng..

[13]  Luca Zaccarian,et al.  Modern Anti-windup Synthesis: Control Augmentation for Actuator Saturation , 2011 .

[14]  J. Nazuno.  Haykin, Simon. Neural networks: A comprehensive foundation, Prentice Hall, Inc. Segunda Edición, 1999 , 2000 .

[15]  A. Kai Qin,et al.  Evolutionary extreme learning machine , 2005, Pattern Recognit..

[16]  Carlos Eduardo Trabuco Dórea,et al.  Output-feedback controlled-invariant polyhedra for constrained linear systems , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[17]  David Q. Mayne,et al.  Optimized robust control invariance for linear discrete-time systems: Theoretical foundations , 2007, Autom..

[18]  Manfred Morari,et al.  Contractive model predictive control for constrained nonlinear systems , 2000, IEEE Trans. Autom. Control..

[19]  Yasar Becerikli,et al.  Neural Network Implementation in Hardware Using FPGAs , 2006, ICONIP.

[20]  Robert F. Stengel,et al.  Neural Networks for Function Approximation in Nonlinear Control , 1990, 1990 American Control Conference.

[21]  Alex Zheng,et al.  Anti-windup design for internal model control , 1994 .

[22]  Michel Kinnaert,et al.  Conditioning technique, a general anti-windup and bumpless transfer method , 1987, Autom..

[23]  Martin Mönnigmann,et al.  A stabilizing control scheme for linear systems on controlled invariant sets , 2015, Syst. Control. Lett..

[24]  David Q. Mayne,et al.  Constrained model predictive control: Stability and optimality , 2000, Autom..

[25]  Manfred Morari,et al.  A unified framework for the study of anti-windup designs , 1994, Autom..

[26]  Jihong Liu,et al.  A Survey of FPGA-Based Hardware Implementation of ANNs , 2005, 2005 International Conference on Neural Networks and Brain.

[27]  Carlos E. T. Dorea,et al.  SET-INVARIANT ESTIMATORS FOR LINEAR SYSTEMS SUBJECT TO DISTURBANCES AND MEASUREMENT NOISE , 2005 .

[28]  Robert F. Stengel,et al.  Smooth function approximation using neural networks , 2005, IEEE Transactions on Neural Networks.

[29]  F. Blanchini.  Ultimate boundedness control for uncertain discrete-time systems via set-induced Lyapunov functions , 1994, IEEE Trans. Autom. Control..

[30]  Hannu T. Toivonen,et al.  A neural network model predictive controller , 2006 .

[31]  Zarita Zainuddin,et al.  Modified wavelet neural network in function approximation and its application in prediction of time-series pollution data , 2011, Appl. Soft Comput..

[32]  Marconi O. de Almeida,et al.  Neural network fitting for input-output manifolds of online control laws in constrained linear systems , 2014, 2014 IEEE Symposium on Computational Intelligence in Control and Automation (CICA).

[33]  Sasa V. Rakovic,et al.  Parameterized Robust Control Invariant Sets for Linear Systems: Theoretical Advances and Computational Remarks , 2010, IEEE Transactions on Automatic Control.

[34]  Dimitry Gorinevsky.  Adaptive learning control using affine radial basis function network approximation , 1993, Proceedings of 8th IEEE International Symposium on Intelligent Control.

[35]  Song-Shyong Chen,et al.  Robust TSK fuzzy modeling for function approximation with outliers , 2001, IEEE Trans. Fuzzy Syst..

[36]  C.E.T. Dorea.  Set-Invariant Estimators for Multiple-Output Discrete-Time Systems , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[37]  J. Hennet,et al.  (A, B)-Invariant Polyhedral Sets of Linear Discrete-Time Systems , 1999 .

[38]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[39]  Nam Mai-Duy,et al.  Approximation of function and its derivatives using radial basis function networks , 2003 .

[40]  Jonathan Parri,et al.  Trigonometric function approximation neural network based coprocessor , 2009 .

[41]  David Q. Mayne,et al.  Robust model predictive control of constrained linear systems with bounded disturbances , 2005, Autom..

[42]  Frank L. Lewis,et al.  Neural-network approximation of piecewise continuous functions: application to friction compensation , 2002, IEEE Trans. Neural Networks.

[43]  K. Keesman,et al.  Nonlinear set-membership estimation: A support vector machine approach , 2004 .

[44]  P. S. Lewis,et al.  Function approximation and time series prediction with neural networks , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[45]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[46]  C. Dórea,et al.  Design of Set-Invariant Estimators for Linear Discrete-Time Systems , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[47]  José Mario Araújo,et al.  Design of Observers With Error Limitation in Discrete-Time Descriptor Systems: A Case Study of a Hydraulic Tank System , 2012, IEEE Transactions on Control Systems Technology.

[48]  W. A. Clark,et al.  Simulation of self-organizing systems by digital computer , 1954, Trans. IRE Prof. Group Inf. Theory.

[49]  M. Morari,et al.  Geometric Algorithm for Multiparametric Linear Programming , 2003 .

[50]  Kevin M. Passino,et al.  Biomimicry for Optimization, Control and Automation , 2004, IEEE Transactions on Automatic Control.

[51]  G. Bitsoris,et al.  Constrained regulation of linear continuous-time dynamical systems , 1989 .

[52]  Xin-Ping Guan,et al.  Output Feedback Stabilization for Time-Delay Nonlinear Interconnected Systems Using Neural Networks , 2008, IEEE Transactions on Neural Networks.

[53]  Kok Kiong Tan,et al.  Intelligent Friction Modeling and Compensation Using Neural Network Approximations , 2012, IEEE Transactions on Industrial Electronics.

[54]  E Mjolsness,et al.  Machine learning for science: state of the art and future prospects. , 2001, Science.

[55]  Renato Spigler,et al.  Approximation by series of sigmoidal functions with applications to neural networks , 2015 .

[56]  Douglas Fisher,et al.  Machine Learning Approaches to Estimating Software Development Effort , 1995, IEEE Trans. Software Eng..

[57]  Kumpati S. Narendra,et al.  Identification and control of dynamical systems using neural networks , 1990, IEEE Trans. Neural Networks.

[58]  Richard J. Fateman,et al.  Lookup tables, recurrences and complexity , 1989, ISSAC '89.

[59]  Richard S. Sutton,et al.  Neural networks for control , 1990 .

[60]  J. M. Araújo,et al.  Dynamic output feedback control of constrained descriptor systems , 2013 .

[61]  José Mario Araújo,et al.  Output feedback control of constrained descriptor systems: a case study of a hydraulic tank system , 2012, J. Syst. Control. Eng..

[62]  Herbert A. Simon,et al.  Applications of machine learning and rule induction , 1995, CACM.

[63]  G. Goodwin,et al.  Set-valued observers for constrained state estimation of discrete-time systems with quantized measurements , 2004, 2004 5th Asian Control Conference (IEEE Cat. No.04EX904).