Simple example of dual control problem with almost analytical solution

Example of dual control of linear uncertain system have been presented. The control task with short horizon (N=2) were solved using dynamic programming. It was shown that the optimal solution is ambiguous, the cost function is non-convex and has many local minima. Optimal control depends in a discontinuous manner on the initial conditions. It was also observed that active learning occurs only when the uncertainty of the initial state exceeds a certain threshold. In this case, the amount of information transmitted from sensor to the controller is much greater than in the case of passive learning.

[1]  Edward Kozłowski,et al.  Comparison of stochastic optimal controls with different level of self-learning , 2006, Ann. UMCS Informatica.

[2]  Robert Tenno,et al.  Dual adaptive controls for linear system with unknown constant parameters , 2010, Int. J. Control.

[3]  Bjarne A. Foss,et al.  Dual adaptive model predictive control , 2017, Autom..

[4]  G. Saridis.  Entropy formulation of optimal and adaptive control , 1988 .

[5]  Peilin Fu,et al.  Optimal nominal dual control for discrete-time LQG problem with unknown parameters , 2003, SICE 2003 Annual Conference (IEEE Cat. No.03TH8734).

[6]  Jie Chen,et al.  Towards Integrating Control and Information Theories , 2017 .

[7]  Maciej Patan,et al.  D-optimal spatio-temporal sampling design for identification of distributed parameter systems , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[8]  K. Loparo,et al.  Optimal state estimation for stochastic systems: an information theoretic approach , 1997, IEEE Trans. Autom. Control..

[9]  Raymond Rishel.  A comment on a dual control problem , 1980 .

[10]  Björn Wittenmark,et al.  Adaptive Dual Control Methods: An Overview , 1995 .

[11]  W. Fleming,et al.  Optimal Control for Partially Observed Diffusions , 1982 .

[12]  Y. Bar-Shalom,et al.  Wide-sense adaptive dual control for nonlinear stochastic systems , 1973 .

[13]  T. Banek,et al.  Control and Cybernetics Incremental Value of Information for Discrete-time Partially Observed Stochastic Systems * , 2022 .

[14]  E. Tse,et al.  Actively adaptive control for nonlinear stochastic systems , 1976, Proceedings of the IEEE.

[15]  Y. Bar-Shalom.  Stochastic dynamic programming: Caution and probing , 1981 .

[16]  K. Loparo,et al.  Optimal control of unknown parameter systems , 1989 .

[17]  J. Sternby.  A simple dual control problem with an analytical solution , 1976 .

[18]  Yaakov Bar-Shalom,et al.  An actively adaptive control for linear systems with random parameters via the dual control approach , 1972, CDC 1972.

[19]  B. Bernhardsson.  Dual Control of a First-Order System with Two Possible Gains , 1989 .

[20]  Edward Kozłowski,et al.  The self-learning active problem in dynamic systems , 2007, Ann. UMCS Informatica.

[21]  Seth Lloyd,et al.  Information-theoretic approach to the study of control systems , 2001, physics/0104007.

[22]  Kunal Kumar,et al.  Experimental Evaluation of a MIMO Adaptive Dual MPC , 2015 .

[23]  Tansu Alpcan,et al.  Learning and information for dual control , 2013, 2013 9th Asian Control Conference (ASCC).

[24]  Alain Bensoussan,et al.  Maximum principle and dynamic programming approaches of the optimal control of partially observed diffusions , 1983 .

[25]  Torsten Bohlin.  Optimal Dual Control of a Simple Process with Unknown Gain , 1969 .

[26]  K. Åström,et al.  Problems of Identification and Control , 1971 .

[27]  K. Åström,et al.  Dual Control of an Integrator with Unknown Gain , 1986 .

[28]  D. Naidu,et al.  Optimal Control Systems , 2018 .

[29]  Bengt Lindoff,et al.  Analysis of approximations of dual control , 1999 .

[30]  Fucai Qian,et al.  Optimal nominal dual control for discrete-time linear-quadratic Gaussian problems with unknown parameters , 2008, Autom..

[31]  Yaakov Bar-Shalom,et al.  Caution, Probing, and the Value of Information in the Control of Uncertain Systems , 1976 .

[32]  Kenneth A. Loparo,et al.  Discrete-time entropy formulation of optimal and adaptive control problems , 1992 .

[33]  Tansu Alpcan,et al.  An Information-Based Learning Approach to Dual Control , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[34]  Tamer Basar,et al.  Dual Control Theory , 2001 .

[35]  K. Loparo,et al.  Dual control of linear stochastic systems with unknown parameters , 1991, IEEE 1991 International Conference on Systems Engineering.

[36]  N. Filatov,et al.  Survey of adaptive dual control methods , 2000 .

[37]  Takahiro Sagawa,et al.  Role of mutual information in entropy production under information exchanges , 2013, 1307.6092.

[38]  Touchette,et al.  Information-theoretic limits of control , 1999, Physical review letters.

[39]  George N. Saridis.  Entropy in Control Engineering , 2001, Series in Intelligent Control and Intelligent Automation.

[40]  Edward Kozłowski,et al.  Active and passive learning in control processes application of the entropy concept , 2005 .

[41]  Edward Kozłowski,et al.  Adaptive control of system entropy , 2006 .

[42]  Edison Tse,et al.  Adaptive Dual Control Methods , 1974 .

[43]  Fucai Qian,et al.  Exact optimal solution for a class of dual control problems , 2016, Int. J. Syst. Sci..

[44]  Piotr Bania,et al.  Field Kalman Filter and its approximation , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).