Hamilton}Jacobi}Bellman framework for optimal control in multistage energy systems

Abstract We enunciate parallelism for structures of variational principles in mechanics and thermodynamics in terms of the duality for thermoeconomic problems of maximizing of production profit and net profit which can be transferred to duality for least action and least abbreviated action which appear in mechanics. With the parallelism in mind, we review theory and macroscopic applications of a recently developed discrete formalism of Hamilton–Jacobi type which arises when Bellman's method of dynamic programming is applied to optimize active (work producing) and inactive (entropy generating) multistage energy systems with free intervals of an independent variable. Our original contribution develops a generalized theory for discrete processes in which these intervals can reside in the model inhomogeneously and can be constrained. We consider applications to multistage thermal machines, controlled unit operations, spontaneous relaxations, nonlinear heat conduction, and self-propagating reaction–diffusion fronts. They all satisfy a basic functional equation that leads to the Hamilton–Jacobi–Bellman equation (HJB equation) and a related discrete optimization algorithm with a maximum principle for a Hamiltonian. Correspondence is shown with the well-known HJB theory for continuous processes when the number of stages approaches an infinity. We show that a common unifying criterion, which is the criterion of a minimum generated entropy, can be proven to act locally in the majority of considered cases, although the related global statements can be invalid far from equilibrium. General limits are found which bound the consumption of the classical work potential (exergy) for finite durations.