Risk-Sensitive Linear Control for Systems With Stochastic Parameters

A novel risk-sensitive (RS) control law is proposed for linear systems with stochastic system parameters. RS control laws are efficient means of handling risk in various failures in control caused by stochastic disturbances. However, stochastic parameters invoke problems, resulting controllers may become nonlinear in the state variable and incompatible with linear systems, cannot be obtained in an exact sense, and are defined only on a bounded state region. To solve these problems, this paper presents a risk-sensitive linear (RSL) control method. The important idea is that a standard RS-type cost function is converted to an expectation of a weighted cost function such that the resulting optimal controller is linear in the state. The weight is designed such that the weighted cost function preserves the characteristics of the original RS control. The designed weight allows one to derive the proposed RSL controller whose exact solution is obtained as a linear feedback law for all states. Furthermore, the RSL control law over an infinite horizon guarantees stochastic stability of the feedback system with the control law. The effectiveness of the proposed RSL control law is demonstrated by a numerical simulation.

[1]  Yasumasa Fujisaki,et al.  Guaranteed cost regulator design: A probabilistic solution and a randomized algorithm , 2007, Autom..

[2]  W. L. De Koning,et al.  Infinite horizon optimal control of linear discrete time systems with stochastic parameters , 1982, Autom..

[3]  Hilbert J. Kappen,et al.  Risk Sensitive Path Integral Control , 2010, UAI.

[4]  Andrew E. B. Lim,et al.  A new risk-sensitive maximum principle , 2005, IEEE Transactions on Automatic Control.

[5]  Yann LeCun,et al.  Large Scale Online Learning , 2003, NIPS.

[6]  Raktim Bhattacharya,et al.  Linear quadratic regulation of systems with stochastic parameter uncertainties , 2009, Autom..

[7]  Kenji Fujimoto,et al.  On stochastic optimal control for linear systems with robust stability , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[8]  H. Nagai.  Bellman equations of risk sensitive control , 1995, Proceedings of 1995 34th IEEE Conference on Decision and Control.

[9]  Rhodes,et al.  Optimal stochastic linear systems with exponential performance criteria and their relation to deterministic differential games , 1973 .

[10]  Andrzej Ruszczynski,et al.  Risk-averse dynamic programming for Markov decision processes , 2010, Math. Program..

[11]  Kenji Fujimoto,et al.  Variance Suppression for Gauge Control via Time-varying Stochastic Optimal Control , 2013 .

[12]  Klaus Obermayer,et al.  A unified framework for risk-sensitive Markov control processes , 2014, 53rd IEEE Conference on Decision and Control.

[13]  James E. Gentle,et al.  Matrix Algebra: Theory, Computations, and Applications in Statistics , 2007 .

[14]  P. McLane.  Optimal stochastic control of linear systems with state- and control-dependent disturbances , 1971 .

[15]  E. Todorov,et al.  A UNIFIED THEORY OF LINEARLY SOLVABLE OPTIMAL CONTROL , 2012 .

[16]  B. Anderson,et al.  Optimal control: linear quadratic methods , 1990 .

[17]  H. Kappen,et al.  Path integral control and state-dependent feedback. , 2014, Physical review. E, Statistical, nonlinear, and soft matter physics.

[18]  H. Kappen.  Path integrals and symmetry breaking for optimal control theory , 2005, physics/0505066.

[19]  Tyrone E. Duncan,et al.  Linear-Exponential-Quadratic Gaussian Control , 2013, IEEE Transactions on Automatic Control.

[20]  Kenji Fujimoto,et al.  System identification based on variational Bayes method and the invariance under coordinate transformations , 2011, IEEE Conference on Decision and Control and European Control Conference.

[21]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[22]  G. Ripaccioli,et al.  Stochastic model predictive control with driver behavior learning for improved powertrain control , 2010, 49th IEEE Conference on Decision and Control (CDC).

[23]  Yuhei Ota,et al.  Optimal control of linear systems with stochastic parameters for variance suppression , 2011, IEEE Conference on Decision and Control and European Control Conference.

[24]  Mark W. Schmidt,et al.  A Stochastic Gradient Method with an Exponential Convergence Rate for Finite Training Sets , 2012, NIPS.

[25]  Kenji Fujimoto,et al.  On linear solutions to a class of risk sensitive control for linear systems with stochastic parameters: Infinite time horizon case , 2016, 2016 American Control Conference (ACC).

[26]  Yuhei Ota,et al.  Optimal Control of Linear Systems with Stochastic Parameters for Variance Suppression: The Finite Time Horizon Case , 2011 .

[27]  Kenji Fujimoto,et al.  On linear solutions to a class of risk sensitive control for linear systems with stochastic parameters , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).