Control of singularly perturbed hybrid stochastic systems

We study a class of optimal stochastic control problems involving two different time scales. The fast mode of the system is represented by deterministic state equations whereas the slow mode of the system corresponds to a jump disturbance process. Under a fundamental "ergodicity" property for a class of "infinitesimal control systems" associated with the fast mode, we show that there exists a limit problem which provides a good approximation to the optimal control of the perturbed system. Both the finite- and infinite-discounted horizon cases are considered. We show how an approximate optimal control law can be constructed from the solution of the limit control problem. In the particular case where the infinitesimal control systems possess the so-called turnpike property, i.e., characterized by the existence of global attractors, the limit control problem can be given an interpretation related to a decomposition approach.

[1]  A. Haurie,et al.  Infinite horizon optimal control : deterministic and stochastic systems , 1991 .

[2]  M. Caramanis,et al.  Production control of manufacturing systems with production rate-dependent failure rates , 1994, IEEE Trans. Autom. Control..

[3]  P. Varaiya,et al.  Multilayer control of large Markov chains , 1978 .

[4]  Suresh P. Sethi,et al.  Asymptotic Optimality in Hierarchical Control of Manufacturing Systems under Uncertainty: State of the Art , 1992 .

[5]  Panganamala Ramana Kumar,et al.  Optimality of Zero-Inventory Policies for Unreliable Manufacturing Systems , 1988, Oper. Res..

[6]  Vladimir Gaitsgory,et al.  Limit Hamilton–Jacobi–Isaacs Equations for Singularly Perturbed Zero-Sum Differential Games , 1996 .

[7]  Mark H. A. Davis.  Piecewise‐Deterministic Markov Processes: A General Class of Non‐Diffusion Stochastic Models , 1984 .

[8]  H. Kushner.  Weak Convergence Methods and Singularly Perturbed Stochastic Control and Filtering Problems , 1990 .

[9]  Stanley B. Gershwin,et al.  Short term production scheduling of an automated manufacturing facility , 1984, The 23rd IEEE Conference on Decision and Control.

[10]  W. Fleming,et al.  An Optimal Stochastic Production Planning Problem with Randomly Fluctuating Demand , 1987 .

[11]  H. Kushner.  Numerical Methods for Stochastic Control Problems in Continuous Time , 2000 .

[12]  P. Lions.  Generalized Solutions of Hamilton-Jacobi Equations , 1982 .

[13]  M. Mariton,et al.  Output feedback for a class of linear systems with stochastic jump parameters , 1985 .

[14]  A. Dontchev,et al.  A Tikhonov-type theorem for singularly perturbed differential inclusions , 1996 .

[15]  V. Volosov,et al.  AVERAGING IN SYSTEMS OF ORDINARY DIFFERENTIAL EQUATIONS , 1962 .

[16]  R. Suri,et al.  Time-optimal control of parts-routing in a manufacturing system with failure-prone machines , 1980, 1980 19th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[17]  J. Filar,et al.  Algorithms for singularly perturbed limiting average Markov control problems , 1992 .

[18]  Raymond Rishel.  Controlled wear process: modeling optimal control , 1991 .

[19]  C. SIAMJ.,et al.  ASYMPTOTIC OPTIMIZATION OF A NONLINEAR HYBRID SYSTEM GOVERNED BY A MARKOV DECISION PROCESS , 1997 .

[20]  Barry E. Griffiths,et al.  Optimal control of jump-linear gaussian systems† , 1985 .

[21]  W. Wonham.  Random differential equations in control theory , 1970 .

[22]  Z. Artstein,et al.  Tracking Fast Trajectories Along a Slow Dynamics: A Singular Perturbations Approach , 1997 .

[23]  D. Verms.  Optimal control of piecewise deterministic markov process , 1985 .

[24]  Zvi Artstein,et al.  Linear-quadratic tracking of coupled slow and fast targets , 1997, Math. Control. Signals Syst..

[25]  R. O'Malley.  Boundary layer methods for certain nonlinear singularly perturbed optimal control problems , 1974 .

[26]  El-Kébir Boukas,et al.  Manufacturing flow control and preventing maintenance: a stochastic control approach , 1988 .

[27]  Christian van Delft,et al.  A turnpike improvement algorithm for piecewise deterministic control , 1991 .

[28]  E. Altman,et al.  Asymptotic Optimization of a Nonlinear Hybrid System Governed by a Markov Decision Process , 1997 .

[29]  R. Malhamé,et al.  A renewal theoretic analysis of a class of manufacturing systems , 1991 .

[30]  V. Gaitsgory.  Suboptimization of singularly perturbed control systems , 1992 .

[31]  D. Sworder.  Feedback control of a class of linear systems with jump parameters , 1969 .

[32]  Fabio Bagagiolo,et al.  Singular Perturbation of a Finite Horizon Problem with State-Space Constraints , 1998 .

[33]  Götz Grammel.  Singularly perturbed differential inclusions: An averaging approach , 1996 .

[34]  Vladimir Gaitsgory.  Suboptimal control of singularly perturbed systems and periodic optimization , 1993, IEEE Trans. Autom. Control..

[35]  Christian van Delft,et al.  Turnpike properties for a class of piecewise deterministic systems arising in manufacturing flow control , 1989, Ann. Oper. Res..

[36]  A. Sharifnia,et al.  Production control of a manufacturing system with multiple machine states , 1988 .

[37]  El-Kébir Boukas,et al.  An optimal control problem with a random stopping time , 1988 .

[38]  Christian van Delft,et al.  Turnpikes in flow control models of unreliable manufacturing systems , 1995 .

[39]  M. K. Ghosh,et al.  Optimal control of switching diffusions with application to flexible manufacturing systems , 1993 .

[40]  Łukasz Stettner,et al.  On ergodic control problems for singularly perturbed Markov processes , 1989 .

[41]  Vladimir Gaitsgory,et al.  On the construction of asymptoticallly optimal controls for singularly perturbed systems , 1997 .

[42]  Roland P. Malhamé,et al.  Ergodicity of hedging control policies in single-part multiple-state manufacturing systems , 1993, IEEE Trans. Autom. Control..

[43]  R. Rishel.  Dynamic Programming and Minimum Principles for Systems with Jump Markov Disturbances , 1975 .

[44]  Zvi Artstein,et al.  Singularly perturbed ordinary differential equations with dynamic limits , 1996, Proceedings of the Royal Society of Edinburgh: Section A Mathematics.

[45]  Eitan Altman,et al.  Control of a hybrid stochastic system , 1993 .

[46]  Michael C. Caramanis,et al.  Near optimal manufacturing flow controller design , 1991 .

[47]  V. G. Gaitsgori,et al.  Theory of Suboptimal Decisions , 1988 .

[48]  Engin Yaz.  Stabilizing control of discrete systems with jump Markov parameters , 1988, Proceedings of the 27th IEEE Conference on Decision and Control.

[49]  G. Grammel.  Averaging of singularly perturbed systems , 1997 .

[50]  H. Soner.  Singular perturbations in manufacturing , 1993 .

[51]  Jerzy A. Filar,et al.  Singularly perturbed Markov control problem: Limiting average cost , 1991 .

[52]  P. Kokotovic.  Applications of Singular Perturbation Techniques to Control Problems , 1984 .

[53]  Alain Haurie,et al.  Boundedly optimal control of piecewise deterministic systems , 1994 .

[54]  A. Vigodner.  Limits of Singularly Perturbed Control Problems with Statistical Dynamics of Fast Motions , 1997 .

[55]  J. Filar,et al.  Perturbation and stability theory for Markov control problems , 1992 .

[56]  Yaakov Bar-Shalom,et al.  A Stabilizing Controller for Jump Linear Gaussian Systems with Noisy State Observations , 1996, Eur. J. Control.

[57]  P. Kokotovic,et al.  A singular perturbation approach to modeling and control of Markov chains , 1981 .

[58]  A. Dontchev.  Perturbations, Approximations, and Sensitivity Analysis of Optimal Control Systems , 1983 .

[59]  François Delebecque,et al.  Optimal control of markov chains admitting strong and weak interactions , 1981, Autom..

[60]  von Weizäscker,et al.  Existence of Optimal Programs of Accumulation for an Infinite Time Horizon , 1965 .

[61]  Petar V. Kokotovic,et al.  Singular perturbations and order reduction in control theory - An overview , 1975, Autom..

[62]  George Liberopoulos,et al.  Perturbation Analysis for the Design of Flexible Manufacturing System Flow Controllers , 1992, Oper. Res..

[63]  A. Haurie,et al.  Infinite Horizon Optimal Control , 1987 .

[64]  R. Akella,et al.  Optimal control of production rate in a failure prone manufacturing system , 1985, 1985 24th IEEE Conference on Decision and Control.

[65]  Stanley B. Gershwin,et al.  Performance of hierarchical production scheduling policy , 1984 .

[66]  Suresh P. Sethi,et al.  Hierarchical controls in stochastic manufacturing systems with machines in tandem , 1992 .