Control and Machine Intelligence for System Autonomy

Autonomous systems evolve from control systems by adding functionalities that increase the level of system autonomy. It is very important to the research in the field that autonomy be well defined and so in the present paper a precise, useful definition of autonomy is introduced and discussed. Autonomy is defined as the ability of the system to attain a set of goals under a set of uncertainties. This leads to the notion of degrees or levels of autonomy. The Quest for Autonomy in engineered systems throughout the centuries is noted, connections to research work of 30 years ago are made and a hierarchical functional architecture for autonomous systems together with needed functionalities are outlined. Adaptation and Learning, which are among the most important functions in achieving high levels of autonomy are then highlighted and recent research contributions are briefly discussed.

[1]  Eric N. Johnson,et al.  Theory and Flight-Test Validation of a Concurrent-Learning Adaptive Controller , 2011 .

[2]  Ke Zhang,et al.  Observer‐based Fault Estimators Using Iterative Learning Scheme for Linear Time‐delay Systems with Intermittent Faults , 2017 .

[3]  Kenji Doya,et al.  Reinforcement Learning in Continuous Time and Space , 2000, Neural Computation.

[4]  K.M. Passino,et al.  Autonomous Control Systems: Architecture and Fundamental Issues , 1988, 1988 American Control Conference.

[5]  Angelia Nedić,et al.  Fast Convergence Rates for Distributed Non-Bayesian Learning , 2015, IEEE Transactions on Automatic Control.

[6]  Panos J. Antsaklis,et al.  Towards intelligent autonomous control systems: Architecture and fundamental issues , 1989, J. Intell. Robotic Syst..

[7]  Cong Wang,et al.  Dynamic Learning From Adaptive Neural Network Control of a Class of Nonaffine Nonlinear Systems , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[8]  B. Shafai,et al.  Magnetic bearing control systems and adaptive forced balancing , 1994, IEEE Control Systems.

[9]  Richard S. Sutton,et al.  Reinforcement Learning is Direct Adaptive Optimal Control , 1992, 1991 American Control Conference.

[10]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[11]  Frank L. Lewis,et al.  Optimal Adaptive Control and Differential Games by Reinforcement Learning Principles , 2012 .

[12]  John R. Anderson,et al.  MACHINE LEARNING An Artificial Intelligence Approach , 2009 .

[13]  King-Sun Fu,et al.  Learning control systems--Review and outlook , 1970 .

[14]  Thomas A. Runkler,et al.  Particle swarm optimization for generating interpretable fuzzy reinforcement learning policies , 2016, Eng. Appl. Artif. Intell..

[15]  J. Sklansky,et al.  Learning systems for automatic control , 1966 .

[16]  Barbara Hayes-Roth,et al.  Intelligent Control , 1994, Artif. Intell..

[17]  Frank L. Lewis,et al.  H∞ control of linear discrete-time systems: Off-policy reinforcement learning , 2017, Autom..

[18]  Andrew G. Barto,et al.  Learning to Act Using Real-Time Dynamic Programming , 1995, Artif. Intell..

[19]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[20]  Frank Allgöwer,et al.  Semi-explicit MPC based on subspace clustering , 2017, Autom..

[21]  P.J. Antsaklis,et al.  Intelligent Learning Control , 1995, IEEE Control Systems.

[22]  Ya Tsypkin,et al.  Self-learning--What is it? , 1968 .

[23]  Frank L. Lewis,et al.  Optimal Control , 1986 .

[24]  R. Babuška,et al.  Port-Hamiltonian Systems in Adaptive and Learning Control: A Survey , 2016, IEEE Transactions on Automatic Control.

[25]  Panos J. Antsaklis,et al.  Introduction to intelligent control systems with high degrees of autonomy , 1993 .

[26]  Jacques Periaux,et al.  Multi-input genetic algorithm for experimental optimization of the reattachment downstream of a backward-facing step with surface plasma actuator , 2015 .

[27]  Farzad R. Salmasi,et al.  Detection of false data injection attacks against state estimation in smart grids based on a mixture Gaussian distribution learning method , 2017, IET Cyper-Phys. Syst.: Theory & Appl..

[28]  Zongmin Ma,et al.  Computers and Games , 2008, Lecture Notes in Computer Science.

[29]  Girish Chowdhary,et al.  Concurrent Learning for Parameter Estimation Using Dynamic State-Derivative Estimators , 2015, IEEE Transactions on Automatic Control.

[30]  Derong Liu,et al.  Adaptive dynamic programming for robust neural control of unknown continuous-time non-linear systems , 2017 .

[31]  L. Gordon Kraft,et al.  A summary comparison of CMAC neural network and traditional adaptive control systems , 1990 .

[32]  Dimitris C. Dracopoulos,et al.  Genetic Algorithms and Genetic Programming for Control , 1997 .

[33]  Panos J. Antsaklis,et al.  Editorial Control Systems and the Quest for Autonomy , 2017, IEEE Trans. Autom. Control..

[34]  Biao Huang,et al.  Constrained data-driven optimal iterative learning control , 2017 .

[35]  Frank L. Lewis,et al.  Optimal and Autonomous Control Using Reinforcement Learning: A Survey , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[36]  Frank L. Lewis,et al.  Optimal Control: Lewis/Optimal Control 3e , 2012 .

[37]  Xuan Yang,et al.  Reinforced gradient-type iterative learning control for discrete linear time-invariant systems with parameters uncertainties and external noises , 2016, IMA J. Math. Control. Inf..

[38]  Michail G. Lagoudakis,et al.  Least-Squares Methods in Reinforcement Learning for Control , 2002, SETN.

[39]  R. Goodman,et al.  Application of neural networks to turbulence control for drag reduction , 1997 .

[40]  Frank L. Lewis,et al.  2009 Special Issue: Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems , 2009 .

[41]  Jing Na,et al.  Global adaptive tracking control of robot manipulators using neural networks with finite-time learning convergence , 2017 .

[42]  John N. Tsitsiklis,et al.  Neuro-dynamic programming: an overview , 1995, Proceedings of 1995 34th IEEE Conference on Decision and Control.

[43]  Ameet Talwalkar,et al.  Foundations of Machine Learning , 2012, Adaptive computation and machine learning.

[44]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[45]  Paul J. Werbos,et al.  Neural networks for control and system identification , 1989, Proceedings of the 28th IEEE Conference on Decision and Control,.

[46]  Mieczyslaw M. Kokar,et al.  Machine learning in a dynamic world , 1988, Proceedings IEEE International Symposium on Intelligent Control 1988.

[47]  Karl Johan Åström,et al.  Control of complex systems , 2001 .

[48]  Xiang Li,et al.  Online model regression for nonlinear time-varying manufacturing systems , 2017, Autom..

[49]  F. Lewis,et al.  Reinforcement Learning and Feedback Control: Using Natural Decision Methods to Design Optimal Adaptive Controllers , 2012, IEEE Control Systems.

[50]  Z. Michalewicz,et al.  A modified genetic algorithm for optimal control problems , 1992 .

[51]  Maarten Steinbuch,et al.  Learning-based identification and iterative learning control of direct-drive robots , 2005, IEEE Transactions on Control Systems Technology.

[52]  Karl Johan Åström,et al.  Adaptive Control , 1989, Embedded Digital Control with Microcontrollers.

[53]  James J. Filliben,et al.  A Scalable Sampling Method to High-Dimensional Uncertainties for Optimal and Reinforcement Learning-Based Controls , 2017, IEEE Control Systems Letters.

[54]  Panos J. Antsaklis,et al.  An introduction to autonomous control systems , 1991 .