From model-based control to data-driven control: Survey, classification and perspective

This paper is a brief survey on the existing problems and challenges inherent in model-based control (MBC) theory, and some important issues in the analysis and design of data-driven control (DDC) methods are here reviewed and addressed. The necessity of data-driven control is discussed from the aspects of the history, the present, and the future of control theories and applications. The state of the art of the existing DDC methods and applications are presented with appropriate classifications and insights. The relationship between the MBC method and the DDC method, the differences among different DDC methods, and relevant topics in data-driven optimization and modeling are also highlighted. Finally, the perspective of DDC and associated research topics are briefly explored and discussed.

[1]  Xuhui Bu,et al.  The robust stability of model free adaptive control with data dropouts , 2010, IEEE ICCA 2010.

[2]  Cheng Qi-ming,et al.  Simulation Study on Model Free Adaptive Control Based on Grey Prediction in Ball Mill Load System , 2010, 2010 International Conference on Artificial Intelligence and Computational Intelligence.

[3]  Biao Huang,et al.  Dynamic Modeling, Predictive Control and Performance Monitoring: A Data-driven Subspace Approach , 2008 .

[4]  Brian D. O. Anderson,et al.  Challenges of adaptive control-past, permanent and future , 2008, Annu. Rev. Control..

[5]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[6]  Z. Hou,et al.  Dual-stage Optimal Iterative Learning Control for Nonlinear Non-affine Discrete-time Systems , 2007 .

[7]  Andrew W. Moore,et al.  Locally Weighted Learning for Control , 1997, Artificial Intelligence Review.

[8]  K. Yubai,et al.  Direct design of switching control system by VRFT -application to vertical-type one-link arm- , 2007, SICE Annual Conference 2007.

[9]  N. S. Khalid.  On-and off-line identification of linear state space models , 2012 .

[10]  Michel Gevers,et al.  Prefiltering in iterative feedback tuning: optimization of the prefilter for accuracy , 2004, IEEE Transactions on Automatic Control.

[11]  W. Gelletly,et al.  New results on , 1996 .

[12]  J. Spall.  Multivariate stochastic approximation using a simultaneous perturbation gradient approximation , 1992 .

[13]  S. Kissling,et al.  Application of iterative feedback tuning (IFT) to speed and position control of a servo drive , 2009 .

[14]  E. K. Gatcombe.  Discussion: “The Measurement of Oil-Film Thickness in Gear Teeth” (MacConochie, I. O., and Cameron, A., 1960, ASME J. Basic Eng., 82, pp. 29–34) , 1960 .

[15]  Shankar P. Bhattacharyya,et al.  New results on the synthesis of PID controllers , 2002, IEEE Trans. Autom. Control..

[16]  Hou Zhong,et al.  On Data-driven Control Theory:the State of the Art and Perspective , 2009 .

[17]  Michael G. Safonov,et al.  The unfalsified control concept: A direct path from experiment to controller , 1995 .

[18]  Michel Gevers,et al.  Modelling, Identification and Control , 2002 .

[19]  L. Valavani,et al.  Robustness of adaptive control algorithms in the presence of unmodeled dynamics , 1982, 1982 21st IEEE Conference on Decision and Control.

[20]  Joon-Mook Lim,et al.  Designing guide-path networks for automated guided vehicle system by using the Q-learning technique , 2003 .

[21]  Jin Soo Lee,et al.  An iterative learning control theory for a class of nonlinear dynamic systems , 1992, Autom..

[22]  Kevin L. Moore,et al.  Iterative Learning Control: Brief Survey and Categorization , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[23]  Maarten Steinbuch,et al.  Data-driven multivariable controller design using Ellipsoidal Unfalsified Control , 2007, 2007 American Control Conference.

[24]  Wan Shu-yun.  The Parameter Identification of PM Synchronous Motor , 2005 .

[25]  Jingwen Yan,et al.  An iterative learning approach for density control of freeway traffic flow via ramp metering , 2008 .

[26]  Frank L. Lewis,et al.  Model-free H∞ control design for unknown linear discrete-time systems via Q-learning with LMI , 2010, Autom..

[27]  Jong-Hwan Kim,et al.  Modular Q-learning based multi-agent cooperation for robot soccer , 2001, Robotics Auton. Syst..

[28]  Wenhu Huang,et al.  The Model-Free Learning Adaptive Control of a Class of Miso Nonlinear Discrete-Time Systems , 1998 .

[29]  Daniel E. Rivera,et al.  A 'Model-on-Demand' identification methodology for non-linear process systems , 2001 .

[30]  Sergio M. Savaresi,et al.  Virtual reference feedback tuning: a direct method for the design of feedback controllers , 2002, Autom..

[31]  Michael J. Grimble,et al.  Iterative Learning Control for Deterministic Systems , 1992 .

[32]  Abdesselem Boulkroune,et al.  Design of a fuzzy adaptive controller for MIMO nonlinear time-delay systems with unknown actuator nonlinearities and unknown control direction , 2010, Inf. Sci..

[33]  Leslie Pack Kaelbling,et al.  Practical Reinforcement Learning in Continuous Spaces , 2000, ICML.

[34]  Editorial Lazy Learning , .

[35]  Michael G. Safonov.  Data-Driven Robust Control Design: Unfalsified Control , 2003 .

[36]  van Jjm Jeroen Helvoort,et al.  Unfalsified control : data-driven control design for performance improvement , 2007 .

[37]  Ying Tan,et al.  Iterative learning control and repetitive control , 2011, Int. J. Control.

[38]  Wallace E. Larimore,et al.  Statistical optimality and canonical variate analysis system identification , 1996, Signal Process..

[39]  Arvin Dehghani,et al.  HISTORICAL, GENERIC AND CURRENT CHALLENGES OF ADAPTIVE CONTROL , 2007 .

[40]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[41]  Sergio M. Savaresi,et al.  Data-driven control design for neuroprotheses: a virtual reference feedback tuning (VRFT) approach , 2004, IEEE Transactions on Control Systems Technology.

[42]  Mauro Birattari,et al.  From Linearization to Lazy Learning: A Survey of Divide-and-Conquer Techniques for Nonlinear Control (Invited Paper) , 2005 .

[43]  Brian D. O. Anderson,et al.  Failures of adaptive control theory and their resolution , 2005, Commun. Inf. Syst..

[44]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[45]  Jian-Bo Yang,et al.  New model for system behavior prediction based on belief rule based systems , 2010, Inf. Sci..

[46]  Brian D. O. Anderson,et al.  Iterative Controller Optimization for Nonlinear Systems , 2003 .

[47]  Paul J. Werbos,et al.  Approximate dynamic programming for real-time control and neural modeling , 1992 .

[48]  F Previdi,et al.  Virtual Reference Feedback Tuning (VRFT) of velocity controller in self-balancing industrial manual manipulators , 2010, Proceedings of the 2010 American Control Conference.

[49]  Xuhui Bu,et al.  Model free adaptive control with data dropouts , 2011, Expert Syst. Appl..

[50]  Wang Wei,et al.  A SURVEY OF ADVANCED PID PARAMETER TUNING METHODS , 2000 .

[51]  Maarten Steinbuch,et al.  Direct data-driven recursive controller unfalsification with analytic update , 2007, Autom..

[52]  Hyun-Ku Rhee,et al.  Design and application of model-on-demand predictive controller to a semibatch copolymerization reactor , 2003 .

[53]  Yasumasa Fujisaki,et al.  System Representation and Optimal Control in Input-Output Data Space , 2004 .

[54]  Bart De Moor,et al.  Subspace Identification for Linear Systems: Theory ― Implementation ― Applications , 2011 .

[55]  Jian-Xin Xu,et al.  Freeway Traffic Control Using Iterative Learning Control-Based Ramp Metering and Speed Signaling , 2007, IEEE Transactions on Vehicular Technology.

[56]  M. Nakamoto.  An application of the virtual reference feedback tuning for an MIMO process , 2004, SICE 2004 Annual Conference.

[57]  Michael G. Safonov,et al.  The Comparison of Unfalsified Control and Iterative Feedback Tuning † , 2002 .

[58]  Dominique Bonvin,et al.  Data-driven controller tuning with integrated stability constraint , 2008, 2008 47th IEEE Conference on Decision and Control.

[59]  Svante Gunnarsson,et al.  Iterative feedback tuning: theory and applications , 1998 .

[60]  Wang Qing-feng.  Nonparametric model adaptive control for underwater towed heave compensation system , 2010 .

[61]  J. Spall,et al.  Model-free control of general discrete-time systems , 1993, Proceedings of 32nd IEEE Conference on Decision and Control.

[62]  Giorgio Battistelli,et al.  Multi-model unfalsified adaptive switching supervisory control , 2010, Autom..

[63]  S. Preitl,et al.  Design and Experiments for a Class of Fuzzy Controlled Servo Systems , 2008, IEEE/ASME Transactions on Mechatronics.

[64]  Wang Gang,et al.  Sugarcane leaves dry anaerobic fermentation. , 2011 .

[65]  Giorgio Battistelli,et al.  Unfalsified adaptive switching supervisory control of time varying systems , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[66]  S. Gunnarsson,et al.  A convergent iterative restricted complexity control design scheme , 1994, Proceedings of 1994 33rd IEEE Conference on Decision and Control.

[67]  Min-Sen Chiu,et al.  New results on VRFT design of PID controller , 2008 .

[68]  L. Miskovic.  data-driven controller tuning using the correlation approach , 2006 .

[69]  B. Gao,et al.  A Model-Free Adaptive Control to a Blood Pump Based on Heart Rate , 2011, ASAIO journal.

[70]  W. Zhang,et al.  Adaptive predictive functional control of a class of nonlinear systems. , 2006, ISA transactions.

[71]  Z. Hou,et al.  The model-free learning adaptive control of a class of SISO nonlinear systems , 1997, Proceedings of the 1997 American Control Conference (Cat. No.97CH36041).

[72]  Michel Gevers,et al.  Correlation-based tuning of decoupling multivariable controllers , 2007, Autom..

[73]  S. Schaal,et al.  Robot juggling: implementation of memory-based learning , 1994, IEEE Control Systems.

[74]  Naoki Hayashi,et al.  A Model-less Algorithm for Tracking Control Based on Input-Output Data , 1998 .

[75]  Giorgio Picci,et al.  Identification, adaptation, learning : the science of learning models from data , 1996 .

[76]  Frank L. Lewis,et al.  Guest Editorial Data-Based Control, Modeling, and Optimization , 2011, IEEE Transactions on Neural Networks.

[77]  Karl Johan Åström,et al.  PID Controllers: Theory, Design, and Tuning , 1995 .

[78]  John E. Warnock,et al.  Dynamic modeling , 1977, SIGGRAPH.

[79]  Robert R. Bitmead,et al.  Direct iterative tuning via spectral analysis , 2000, Autom..

[80]  Frank L. Lewis,et al.  Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity control , 2007, Autom..

[81]  Hou Zhongsheng.  On model-free adaptive control:the state of the art and perspective , 2006 .

[82]  J. Sjoberg,et al.  Nonlinear controller tuning based on linearized time-variant model , 1997, Proceedings of the 1997 American Control Conference (Cat. No.97CH36041).

[83]  Pedro Albertos,et al.  Iterative Identification and Control , 2002, Springer London.

[84]  A. E. Graham,et al.  Rapid tuning of controllers by IFT for profile cutting machines , 2007 .

[85]  J. G. Ziegler,et al.  Optimum Settings for Automatic Controllers , 1942, Journal of Fluids Engineering.

[86]  YangQuan Chen,et al.  Iterative Learning Control: Convergence, Robustness and Applications , 1999 .

[87]  Alex Weissensteiner,et al.  A $Q$ -Learning Approach to Derive Optimal Consumption and Investment Strategies , 2008, IEEE Transactions on Neural Networks.

[88]  Jian-Xin Xu,et al.  Notes on Data-driven System Approaches: Notes on Data-driven System Approaches , 2009 .

[89]  D. C. Chin,et al.  Traffic-responsive signal timing for system-wide traffic control , 1997, Proceedings of the 1997 American Control Conference (Cat. No.97CH36041).

[90]  Juha T. Tanttu,et al.  TUNING OF PID CONROLLERS: SURVEY OF SISO AND MIMO TECHNIQUES , 1991 .

[91]  Jian-Xin Xu,et al.  Iterative Learning Control , 1998 .

[92]  Chris Watkins,et al.  Learning from delayed rewards , 1989 .

[93]  Zhongsheng Hou,et al.  Model-free periodic adaptive control for a class of SISO nonlinear discrete-time systems , 2010, IEEE ICCA 2010.

[94]  Masaru Uchiyama,et al.  Formation of High-Speed Motion Pattern of a Mechanical Arm by Trial , 1978 .

[95]  Ljubisa Miskovic,et al.  Iterative correlation-based controller tuning with application to a magnetic suspension system , 2003 .

[96]  Ljubisa Miskovic,et al.  Correlation-Based Tuning of a Restricted-Complexity Controller for an Active Suspension System , 2003, Eur. J. Control.

[97]  J. Willems,et al.  DATA DRIVEN SIMULATION WITH APPLICATIONS TO SYSTEM IDENTIFICATION , 2005 .

[98]  Alexander S. Poznyak,et al.  Identification of Chemical Processes , 2001 .

[99]  Sergio M. Savaresi,et al.  An Application of the Virtual Reference Feedback Tuning Method to a Benchmark Problem , 2003, Eur. J. Control.

[100]  Vassilis G. Kaburlasos,et al.  Piecewise-linear approximation of non-linear models based on probabilistically/possibilistically interpreted intervals' numbers (INs) , 2010, Inf. Sci..

[101]  Klaske van Heusden.  Non-Iterative Data-Driven Model Reference Control , 2010 .

[102]  Per-Olof Gutman,et al.  Nonlinear controller tuning based on a sequence of identifications of linearized time-varying models , 2009 .

[103]  Andrew G. Barto,et al.  Adaptive linear quadratic control using policy iteration , 1994, Proceedings of 1994 American Control Conference - ACC '94.

[104]  Jian-Xin Xu,et al.  A New Feedback-feedforward Configuration for the Iterative Learning Control of a Class of Discrete-time Systems , 2007 .

[105]  A. Karimi,et al.  Non-iterative data-driven controller tuning using the correlation approach , 2007, 2007 European Control Conference (ECC).

[106]  Ling Chen,et al.  A clustering algorithm for multiple data streams based on spectral component similarity , 2012, Inf. Sci..

[107]  Sergio M. Savaresi,et al.  Direct nonlinear control design: the virtual reference feedback tuning (VRFT) approach , 2006, IEEE Transactions on Automatic Control.

[108]  Xu Jian.  On Learning Control:The State of the Art and Perspective , 2005 .

[109]  Michel Gevers,et al.  Optimal prefiltering in iterative feedback tuning , 2005, IEEE Transactions on Automatic Control.

[110]  D. Owens.  Iterative learning control-convergence using high gain feedback , 1992, [1992] Proceedings of the 31st IEEE Conference on Decision and Control.

[111]  R. E. Kalman,et al.  Contributions to the Theory of Optimal Control , 1960 .

[112]  Zhongsheng Hou,et al.  Notes on Data-driven System Approaches , 2009 .

[113]  Håkan Hjalmarsson,et al.  Control of nonlinear systems using iterative feedback tuning , 1998, Proceedings of the 1998 American Control Conference. ACC (IEEE Cat. No.98CH36207).

[114]  Chiang-Ju Chien.  A discrete iterative learning control for a class of nonlinear time-varying systems , 1998 .

[115]  Dale E. Seborg,et al.  Identification of chemical processes using canonical variate analysis , 1994 .

[116]  Suguru Arimoto,et al.  Bettering operation of Robots by learning , 1984, J. Field Robotics.

[117]  Sergio M. Savaresi,et al.  Virtual reference direct design method: an off-line approach to data-based control system design , 2000, IEEE Trans. Autom. Control..

[118]  Masao Ikeda,et al.  Stability analysis and control design of LTI discrete-time systems by the direct use of time series data , 2009, Autom..

[119]  Frank L. Lewis,et al.  Special Section on Data-Based Control, Modeling, and Optimization , 2011 .

[120]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[121]  Sabine Van Huffel,et al.  Exact and Approximate Modeling of Linear Systems: A Behavioral Approach (Mathematical Modeling and Computation) (Mathematical Modeling and Computation) , 2006 .

[122]  Hou Zhong,et al.  Model Free Adaptive Control Based Freeway Ramp Metering with Feedforward Iterative Learning Controller , 2009 .

[123]  Paul J. Webros.  A menu of designs for reinforcement learning over time , 1990 .

[124]  Michael G. Safonov,et al.  The unfalsified control concept and learning , 1997 .

[125]  H. Hjalmarsson.  Efficient tuning of linear multivariable controllers using iterative feedback tuning , 1999 .

[126]  Bin Gao,et al.  An anti-suction control for an intra-aorta pump using blood assistant index: a numerical simulation. , 2012, Artificial organs.

[127]  J. Spall.  Adaptive stochastic approximation by the simultaneous perturbation method , 1998, Proceedings of the 37th IEEE Conference on Decision and Control (Cat. No.98CH36171).

[128]  Michel Verhaegen,et al.  Identification of the deterministic part of MIMO state space models given in innovations form from input-output data , 1994, Autom..

[129]  Robert E. Skelton,et al.  Model error concepts in control design , 1989 .

[130]  Ljubisa Miskovic,et al.  Convergence Analysis of an Iterative Correlation-Based Controller Tuning Method , 2002 .

[131]  Paolo Rapisarda,et al.  Data-driven simulation and control , 2008, Int. J. Control.

[132]  Z. Hou,et al.  On Data-driven Control Theory: the State of the Art and Perspective: On Data-driven Control Theory: the State of the Art and Perspective , 2009 .

[133]  A. Paul,et al.  Cost-detectability and Stability of Adaptive Control Systems , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[134]  H. Ishigaki,et al.  A lazy learning control method using support vector regression , 2007, 2007 Mediterranean Conference on Control & Automation.

[135]  ChaiTianyou,et al.  Guest Editorial Data-Based Control, Modeling, and Optimization , 2011 .

[136]  Maarten Steinbuch,et al.  Data-Driven Controller Unfalsification With Analytic Update Applied to a Motion System , 2008, IEEE Transactions on Control Systems Technology.

[137]  David W. Aha,et al.  Instance-Based Learning Algorithms , 1991, Machine Learning.

[138]  Jianxin Xu,et al.  Linear and Nonlinear Iterative Learning Control , 2003 .

[139]  James C. Spall.  Feedback and Weighting Mechanisms for Improving Jacobian Estimates in the Adaptive Simultaneous Perturbation Algorithm , 2009, IEEE Trans. Autom. Control..

[140]  Csilla Bányász,et al.  Iterative Identification and Control Design , 2001 .

[141]  Tong-heng Lee,et al.  Adaptive-Predictive Control of a Class of SISO Nonlinear Systems , 2001 .

[142]  Shangtai Jin,et al.  A statistical analysis of model free adaptive control with measurement disturbance , 2010, Proceedings of the 29th Chinese Control Conference.

[143]  Tianhong Pan,et al.  Lazy learning-based online identification and adaptive PID control : A case study for CSTR process , 2007 .

[144]  Fabio Previdi,et al.  Closed-loop control of FES supported standing up and sitting down using Virtual Reference Feedback Tuning , 2005 .

[145]  Michael Athans,et al.  Robustness of continuous-time adaptive control algorithms in the presence of unmodeled dynamics , 1985 .

[146]  N. K. Poulsen,et al.  Improving Convergence of Iterative Feedback Tuning , 2009 .

[147]  J. Spall,et al.  Model-free control of nonlinear stochastic systems with discrete-time measurements , 1998, IEEE Trans. Autom. Control..

[148]  Antonio Sala,et al.  Extensions to "virtual reference feedback tuning: A direct method for the design of feedback controllers" , 2005, Autom..

[149]  Hou Zhongsheng.  Convergence analysis of learning-enhanced PID control system , 2010 .

[150]  Giorgio Battistelli,et al.  Stability of Unfalsified Adaptive Switching Control in Noisy Environments , 2010, IEEE Transactions on Automatic Control.

[151]  George Cybenko,et al.  Just-in-Time Learning and Estimation , 1996 .

[152]  Tore Hägglund,et al.  Automatic Tuning of Pid Controllers , 1988 .

[153]  B. C. Brookes,et al.  Information Sciences , 2020, Cognitive Skills You Need for the 21st Century.

[154]  Shangtai Jin,et al.  Data-Driven Model-Free Adaptive Control for a Class of MIMO Nonlinear Discrete-Time Systems , 2011, IEEE Transactions on Neural Networks.

[155]  Jay H. Lee,et al.  Approximate dynamic programming-based approaches for input-output data-driven control of nonlinear processes , 2005, Autom..

[156]  Mauro Birattari,et al.  Lazy learning for modeling and control design , 1997 .

[157]  Håkan Hjalmarsson,et al.  Iterative feedback tuning—an overview , 2002 .

[158]  Shangtai Jin,et al.  A Novel Data-Driven Control Approach for a Class of Discrete-Time Nonlinear Systems , 2011, IEEE Transactions on Control Systems Technology.

[159]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[160]  Tore Hägglund,et al.  Automatic Tuning and Adaptation for PID Controllers - A Survey , 1992 .

[161]  Huabin Chen,et al.  A novel control algorithm for weld pool control , 2010, Ind. Robot.

[162]  S. Van Huffel,et al.  Exact and Approximate Modeling of Linear Systems: A Behavioral Approach , 2006 .

[163]  Henry Page Croft,et al.  Glucagon: its significance in health and disease. , 1976, The Ulster medical journal.

[164]  Bruce Hannon,et al.  Dynamic Modeling , 1994, Springer US.

[165]  Tohru Katayama,et al.  Subspace Methods for System Identification , 2005 .

[166]  Antonio Sala.  Integrating virtual reference feedback tuning into a unified closed-loop identification framework , 2007, Autom..

[167]  M. Birattari,et al.  Lazy learning for local modelling and control design , 1999 .

[168]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[169]  Leandro dos Santos Coelho,et al.  Model-free adaptive control optimization using a chaotic particle swarm approach , 2009 .