Linear Programming Approach to Deterministic Infinite Horizon Optimal Control Problems with Discounting

We investigate relationships between the deterministic infinite time horizon optimal control problem with discounting, in which the state trajectories remain in a given compact set $Y$, and a certain infinite dimensional linear programming (IDLP) problem. We introduce the problem dual with respect to this IDLP problem and obtain some duality results. We construct necessary and sufficient optimality conditions for the optimal control problem under consideration, and we give a characterization of the viability kernel of $Y$. We also indicate how one can use finite dimensional approximations of the IDLP problem and its dual for construction of near optimal feedback controls. The construction is illustrated with a numerical example.

[1]  Vladimir Gaitsgory,et al.  Averaging and Near Viability of Singularly Perturbed Control Systems , 2006 .

[2]  L. Grüne.  Asymptotic Controllability and Exponential Stabilization of Nonlinear Control Systems at Singular Points , 1998 .

[3]  Asen L. Dontchev,et al.  Singular Perturbations in Infinite-Dimensional Control Systems , 2003, SIAM J. Control. Optim..

[4]  H. Frankowska.  Hamilton-Jacobi equations: Viscosity solutions and generalized gradients , 1989 .

[5]  J. Warga.  Optimal control of differential and functional equations , 1972 .

[6]  R. Stockbridge,et al.  Numerical Comparison of Controls and Verification of Optimality for Stochastic Control Problems , 2000 .

[7]  W. Fleming,et al.  Convex duality approach to the optimal control of diffusions , 1989 .

[8]  M. Bardi,et al.  Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations , 1997 .

[9]  R. Stockbridge.  Time-Average Control of Martingale Problems: A Linear Programming Formulation , 1990 .

[10]  Vladimir Gaitsgory,et al.  Linear Programming Approach to Deterministic Long Run Average Problems of Optimal Control , 2006, SIAM J. Control. Optim..

[11]  Adam M. Oberman,et al.  Computing the Effective Hamiltonian using a Variational Approach , 2004, Proceedings of the 44th IEEE Conference on Decision and Control.

[12]  A. Leizarowitz,et al.  Infinite-Horizon Discrete-Time Optimal Control Problems , 2003 .

[13]  K. Parthasarathy,et al.  Probability measures on metric spaces , 1967 .

[14]  E. Anderson.  Linear Programming In Infinite Dimensional Spaces , 1970 .

[15]  José G. Llavona,et al.  Approximation of Continuously Differentiable Functions , 1986 .

[16]  Qing Zhang,et al.  Continuous-Time Markov Chains and Applications , 1998 .

[17]  F. Rampazzo,et al.  Filippov's and Filippov–Ważewski's Theorems on Closed Domains , 2000 .

[18]  Ludovic Rifford,et al.  Approximation and regularization of Lipschitz functions: convergence of the gradients , 2006 .

[19]  M. Bardi,et al.  Ergodic Problems in Differential Games , 2007 .

[20]  Jacek B. Krawczyk,et al.  Applying a finite-horizon numerical optimization method to a periodic optimal control problem , 2008, Autom..

[21]  R. Stockbridge.  Time-Average Control of Martingale Problems: Existence of a Stationary Solution , 1990 .

[22]  Vivek S. Borkar,et al.  Averaging of Singularly Perturbed Controlled Stochastic Differential Equations , 2007 .

[23]  R. Vinter.  Convex duality and nonlinear optimal control , 1993 .

[24]  W. Rudin.  Real and complex analysis, 3rd ed. , 1987 .

[25]  L. Grüne.  On the Relation between Discounted and Average Optimal Value Functions , 1998 .

[26]  T. Kurtz,et al.  Existence of Markov Controls and Characterization of Optimal Markov Controls , 1998 .

[27]  V. Borkar,et al.  Occupation measures for controlled Markov processes: characterization and optimality , 1996 .

[28]  Hitoshi Ishii,et al.  A Characterization of the Existence of Solutions for Hamilton—Jacobi Equations in Ergodic Control Problems with Applications , 2000 .

[29]  Marc Quincampoix,et al.  Differential inclusions and target problems , 1992 .

[30]  Diogo A. Gomes,et al.  Linear programming interpretations of Mather's variational principle , 2002 .

[31]  B. Craven.  Control and optimization , 2019, Mathematical Modelling of the Human Cardiovascular System.

[32]  Vladimir Gaitsgory,et al.  Duality in Linear Programming Problems Related to Deterministic Long Run Average Problems of Optimal Control , 2008, SIAM J. Control. Optim..

[33]  Yu. S. Ledyaev,et al.  Qualitative properties of trajectories of control systems: A survey , 1995 .

[34]  Dimitri P. Bertsekas,et al.  Stochastic optimal control : the discrete time case , 2007 .

[35]  Gopal K. Basak,et al.  Ergodic control of degenerate diffusions , 1997 .

[36]  Vladimir Gaitsgory,et al.  On a Representation of the Limit Occupational Measures Set of a Control System with Applications to Singularly Perturbed Control Systems , 2004, SIAM J. Control. Optim..

[37]  Z. Artstein.  INVARIANT MEASURES AND THEIR PROJECTIONS IN NONAUTONOMOUS DYNAMICAL SYSTEMS , 2004 .

[38]  A. Bensoussan.  Perturbation Methods in Optimal Control , 1988 .

[39]  W. Fleming,et al.  Controlled Markov processes and viscosity solutions , 1992 .

[40]  C. Villani.  Topics in Optimal Transportation , 2003 .

[41]  O. Hernández-Lerma,et al.  Markov chains and invariant probabilities , 2003 .

[42]  Emmanuel Trélat,et al.  Nonlinear Optimal Control via Occupation Measures and LMI-Relaxations , 2007, SIAM J. Control. Optim..

[43]  Marc Quincampoix,et al.  Stochastic control and compatible subsets of constraints , 2005 .

[44]  O. Hernández-Lerma,et al.  THE LINEAR PROGRAMMING APPROACH TO DETERMINISTIC OPTIMAL CONTROL PROBLEMS , 1996 .

[45]  H. Kushner.  Numerical Methods for Stochastic Control Problems in Continuous Time , 2000 .