Nonlinear Optimal Control via Occupation Measures and LMI-Relaxations

We consider the class of nonlinear optimal control problems (OCPs) with polynomial data, i.e., the differential equation, state and control constraints, and cost are all described by polynomials, and more generally for OCPs with smooth data. In addition, state constraints as well as state and/or action constraints are allowed. We provide a simple hierarchy of LMI- (linear matrix inequality)-relaxations whose optimal values form a nondecreasing sequence of lower bounds on the optimal value. Under some convexity assumptions, the sequence converges to the optimal value of the OCP. Preliminary results show that good approximations are obtained with few moments.

[1]  J. Nash.  The imbedding problem for Riemannian manifolds , 1956 .

[2]  L. S. Pontryagin,et al.  Mathematical Theory of Optimal Processes , 1962 .

[3]  E. Blum,et al.  The Mathematical Theory of Optimal Processes. , 1963 .

[4]  J. Krivine,et al.  Anneaux préordonnés , 1964 .

[5]  Christian Coatmélec.  Approximation et interpolation des fonctions différentiables de plusieurs variables , 1966 .

[6]  D. Jacobson,et al.  New necessary conditions of optimality for control problems with state-variable inequality constraints , 1971 .

[7]  H. Maurer.  On Optimal Control Problems with Bounded State Variables and Control Appearing Linearly , 1975, Optimization Techniques.

[8]  Moshe Ben-Horim,et al.  A linear programming approach , 1977 .

[9]  D. Dawson.  Qualitative behavior of geostochastic systems , 1980 .

[10]  Philip E. Gill,et al.  Practical optimization , 1981 .

[11]  R. W. Brockett,et al.  Asymptotic stability and feedback stabilization , 1982 .

[12]  H. Soner.  Optimal control with state-space constraint I , 1986 .

[13]  C. Berg.  The multidimensional moment problem and semi-groups , 1987 .

[14]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[15]  W. Fleming,et al.  Convex duality approach to the optimal control of diffusions , 1989 .

[16]  Gautam Appa,et al.  Linear Programming in Infinite-Dimensional Spaces , 1989 .

[17]  P. Lions,et al.  Hamilton-Jacobi equations with state constraints , 1990 .

[18]  K. Schmüdgen.  TheK-moment problem for compact semi-algebraic sets , 1991 .

[19]  K. Schmüdgen.  TheK-moment problem for compact semi-algebraic sets , 1991 .

[20]  Oskar von Stryk,et al.  Direct and indirect methods for trajectory optimization , 1992, Ann. Oper. Res..

[21]  R. Vinter.  Convex duality and nonlinear optimal control , 1993 .

[22]  H. J. Pesch.  A Practical Guide to the Solution of Real-Life Optimal Control Problems , 1994 .

[23]  Suresh P. Sethi,et al.  A Survey of the Maximum Principles for Optimal Control Problems with State Constraints , 1995, SIAM Rev..

[24]  O. Hernández-Lerma,et al.  THE LINEAR PROGRAMMING APPROACH TO DETERMINISTIC OPTIMAL CONTROL PROBLEMS , 1996 .

[25]  V. Borkar,et al.  Occupation measures for controlled Markov processes: characterization and optimality , 1996 .

[26]  W. Fleming.  Book Review: Discrete-time Markov control processes: Basic optimality criteria , 1997 .

[27]  T. Kurtz,et al.  Existence of Markov Controls and Characterization of Optimal Markov Controls , 1998 .

[28]  Onésimo Hernández-Lerma,et al.  Approximation Schemes for Infinite Linear Programs , 1998, SIAM J. Optim..

[29]  O. Hernández-Lerma,et al.  Discrete-time Markov control processes , 1999 .

[30]  O. Hernández-Lerma,et al.  Further topics on discrete-time Markov control processes , 1999 .

[31]  Numerical Comparison of Controls and Verification of Optimality for Stochastic Control Problems , 2000 .

[32]  B. Gaveau,et al.  Hamilton–Jacobi theory and the heat kernel on Heisenberg groups , 2000 .

[33]  Kurt Helmes,et al.  Computing Moments of the Exit Time Distribution for Markov Processes by Linear Programming , 2001, Oper. Res..

[34]  Jean B. Lasserre,et al.  Global Optimization with Polynomials and the Problem of Moments , 2000, SIAM J. Optim..

[35]  Arnold Neumaier,et al.  Introduction to Numerical Analysis , 2001 .

[36]  B. Dundas,et al.  DIFFERENTIAL TOPOLOGY , 2002 .

[37]  Vivek S. Borkar,et al.  Convex Analytic Methods in Markov Decision Processes , 2002 .

[38]  Richard H. Stockbridge,et al.  Linear Programming Formulation for Optimal Stopping Problems , 2001, SIAM J. Control. Optim..

[39]  O. Hernández-Lerma,et al.  Markov chains and invariant probabilities , 2003 .

[40]  J. Lasserre,et al.  SDP vs. LP Relaxations for the Moment Approach in Some Performance Evaluation Problems , 2004 .

[41]  J. Lasserre,et al.  Solving nonconvex optimization problems , 2004, IEEE Control Systems.

[42]  Emmanuel Trélat,et al.  Contrôle optimal : théorie & applications , 2005 .

[43]  Emmanuel Trélat,et al.  Robust optimal stabilization of the Brockett integrator via a hybrid feedback , 2005, Math. Control. Signals Syst..

[44]  J. Lasserre,et al.  Nonlinear optimal control: Numerical approximations via moments and LMI-relaxations , 2005 .

[45]  Vladimir Gaitsgory,et al.  Linear Programming Approach to Deterministic Long Run Average Problems of Optimal Control , 2006, SIAM J. Control. Optim..

[46]  R. Munos,et al.  An anti-diffusive scheme for viability problems , 2006 .

[47]  J. Lasserre,et al.  PRICING A CLASS OF EXOTIC OPTIONS VIA MOMENTS AND SDP RELAXATIONS , 2006 .