Discrete Hamilton-Jacobi Theory

We develop a discrete analogue of Hamilton–Jacobi theory in the framework of discrete Hamiltonian mechanics. The resulting discrete Hamilton–Jacobi equation is discrete only in time. We describe a discrete analogue of Jacobi's solution and also prove a discrete version of the geometric Hamilton–Jacobi theorem. The theory applied to discrete linear Hamiltonian systems yields the discrete Riccati equation as a special case of the discrete Hamilton–Jacobi equation. We also apply the theory to discrete optimal control problems, and recover some well-known results, such as the Bellman equation (discrete-time HJB equation) of dynamic programming and its relation to the costate variable in the Pontryagin maximum principle. This relationship between the discrete Hamilton–Jacobi equation and Bellman equation is exploited to derive a generalized form of the Bellman equation that has controls at internal stages.

[1]  Robin Chhabra.  Nonholonomic Hamilton-Jacobi Theory via Chaplygin Hamiltonization , 2011 .

[2]  J. Marsden,et al.  Lie-Poisson Hamilton-Jacobi theory and Lie-Poisson integrators , 1988 .

[3]  Richard Bellman,et al.  Introduction to the mathematical theory of control processes , 1967 .

[4]  C. Scovel,et al.  Symplectic integration of Hamiltonian systems , 1990 .

[5]  Anthony M. Bloch,et al.  A discrete maximum principle for solving optimal control problems , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[6]  V. Arnold.  Mathematical Methods of Classical Mechanics , 1974 .

[7]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[8]  S. Lall,et al.  Discrete variational Hamiltonian mechanics , 2006 .

[9]  J. Marsden,et al.  Introduction to mechanics and symmetry , 1994 .

[10]  R. Brockett.  Control Theory and Singular Riemannian Geometry , 1982 .

[11]  Tomoki Ohsawa,et al.  Nonholonomic Hamilton-Jacobi equation and integrability , 2009, 0906.3357.

[12]  M. León,et al.  Towards a Hamilton–Jacobi theory for nonholonomic mechanical systems , 2007, 0705.3739.

[13]  David K. Smith,et al.  Dynamic Programming and Optimal Control. Volume 1 , 1996 .

[14]  William W. Hager,et al.  Runge-Kutta methods in optimal control and the transformed adjoint system , 2000, Numerische Mathematik.

[15]  J. Moser,et al.  Discrete versions of some classical integrable systems and factorization of matrix polynomials , 1991 .

[16]  J. Marsden,et al.  Discrete mechanics and variational integrators , 2001, Acta Numerica.

[17]  Juan Carlos Marrero,et al.  Linear almost Poisson structures and Hamilton-Jacobi equation. Applications to nonholonomic Mechanics , 2008, 0801.4358.

[18]  J. W. Humberston.  Classical mechanics , 1980, Nature.

[19]  G. Marmo,et al.  Geometric Hamilton-Jacobi theory for nonholonomic dynamical systems , 2009, 0908.2453.

[20]  David L. Elliott,et al.  Geometric control theory , 2000, IEEE Trans. Autom. Control..

[21]  Tomoki Ohsawa,et al.  Hamilton-Jacobi Theory for Degenerate Lagrangian Systems with Holonomic and Nonholonomic Constraints , 2011, 1109.6056.

[22]  Jeremy Schiff,et al.  The Hamilton-Jacobi difference equation , 2004 .

[23]  P J Fox,et al.  THE FOUNDATIONS OF MECHANICS. , 1918, Science.

[24]  Gregory S. Ammar,et al.  The geometry of matrix eigenvalue methods , 1986 .

[25]  Betty Jean Harmsen.  The discrete calculus of variations , 1995 .

[26]  J. Troutman.  Variational Principles in Mechanics , 1983 .

[27]  Jerrold E. Marsden,et al.  Foundations of computational geometric mechanics , 2004 .

[28]  Melvin Leok,et al.  Discrete Hamiltonian Variational Integrators , 2010, 1001.1408.

[29]  S. Brendle,et al.  Calculus of Variations , 1927, Nature.

[30]  B. W. Jordan,et al.  Theory of a Class of Discrete Optimal Control Systems , 1964 .

[31]  Jerrold E. Marsden,et al.  Discrete Routh reduction , 2006 .

[32]  K. Lynch.  Nonholonomic Mechanics and Control , 2004, IEEE Transactions on Automatic Control.

[33]  Y. Suris.  The Problem of Integrable Discretization: Hamiltonian Approach , 2003 .

[34]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[35]  Y. Suris,et al.  Discrete Lagrangian Models , 2004 .