Multiple Model-Based Control Using Finite Controlled Markov Chains

Cognition and control processes share many similar characteristics, and decisionmaking and learning under the paradigm of multiple models has increasingly gained attention in both fields. The controlled finite Markov chain (CFMC) approach enables to deal with a large variety of signals and systems with multivariable, nonlinear, and stochastic nature. In this article, adaptive control based on multiple models is considered. For a set of candidate plant models, CFMC models (and controllers) are constructed off-line. The outcomes of the CFMC models are compared with frequentist information obtained from on-line data. The best model (and controller) is chosen based on the Kullback–Leibler information. This approach to adaptive control emphasizes the use of physical models as the basis of reliable plant identification. Three series of simulations are conducted: to examine the performace of the developed Matlab-tools; to illustrate the approach in the control of a nonlinear nonminimum phase van der Vusse CSTR plant; and to examine the suggested model selection method for the adaptive control.

[1]  C. Hsu,et al.  Cell-To-Cell Mapping A Method of Global Analysis for Nonlinear Systems , 1987 .

[2]  X. Rong Li,et al.  General model-set design methods for multiple-model approach , 2005, IEEE Transactions on Automatic Control.

[3]  John G. Kemeny,et al.  Finite Markov chains , 1960 .

[4]  Vadim Mizonov,et al.  Applications of Markov Chains in Particulate Process Engineering: A Review , 2004 .

[5]  Jan Lunze,et al.  On the Markov Property of Quantised State Measuirement Sequences , 1998, Autom..

[6]  G. Ososkov.  recognition and control , 2003 .

[7]  Dimitar Petrov Filev.  Model bank based intelligent control , 2002, 2002 Annual Meeting of the North American Fuzzy Information Processing Society Proceedings. NAFIPS-FLINT 2002 (Cat. No. 02TH8622).

[8]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[9]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[10]  John G. Kemeny,et al.  Finite Markov Chains. , 1960 .

[11]  C. S. Hsu,et al.  Cell-to-Cell Mapping , 1987 .

[12]  Abhijit Gosavi,et al.  Reinforcement Learning: A Tutorial Survey and Recent Advances , 2009, INFORMS J. Comput..

[13]  Sirish L. Shah,et al.  Recursive least squares based estimation schemes for self‐tuning control , 1991 .

[14]  Jan Lunze,et al.  Process supervision by means of a hybrid model , 2001 .

[15]  Jong Min Lee,et al.  Approximate Dynamic Programming Strategies and Their Applicability for Process Control: A Review and Future Directions , 2004 .

[16]  Alexander S. Poznyak,et al.  Self-Learning Control of Finite Markov Chains , 2000 .

[17]  R. Kulhavý.  A Kullback-Leibler distance approach to system identification , 1996 .

[18]  D. J. White,et al.  A Survey of Applications of Markov Decision Processes , 1993 .

[19]  Tatiana V. Guy,et al.  Cooperative decision making without facilitator , 2007, ALCOSP.

[20]  Robert Shorten,et al.  On the interpretation of local models in blended multiple model structures. , 1999 .

[21]  Roderick Murray-Smith,et al.  Multiple Model Approaches to Modelling and Control , 1997 .

[22]  J. Spruce Riordon,et al.  An adaptive automaton controller for discrete-time markov processes , 1969, Autom..

[23]  D. J. White,et al.  Real Applications of Markov Decision Processes , 1985 .

[24]  A Motter Mark,et al.  Predictive Multiple Model Switching Control With the Self-Organizing Map , 2000 .

[25]  D. J. White,et al.  Further Real Applications of Markov Decision Processes , 1988 .

[26]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[27]  R. Kulhavý.  A Kullback-Leibler distance approach to system identification , 1996 .

[28]  Amir Hussain,et al.  Emergent Common Functional Principles in Control Theory and the Vertebrate Brain: A Case Study with Autonomous Vehicle Control , 2008, ICANN.

[29]  Michael I. Jordan,et al.  Learning from Incomplete Data , 1994 .

[30]  Mark A. Motter,et al.  Predictive multiple model switching control with the self-organizing map , 2002, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[31]  Kaddour Najim,et al.  Advanced Process Identification and Control , 2001 .

[32]  J. S. Riordon.  An adaptive automaton controller for discrete-time Markov processes , 1969 .

[33]  Kumpati S. Narendra,et al.  Adaptation and learning using multiple models, switching, and tuning , 1995 .

[34]  Olle Häggström.  Finite Markov Chains and Algorithmic Applications , 2002 .

[35]  Roderick Murray-Smith,et al.  The operating regime approach to nonlinear modelling and control , 1997 .

[36]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.