Convergence of stochastic approximation coupled with perturbation analysis in a class of manufacturing flow control models

This paper deals with a class of piecewise determinstic control systems for which the optimal control can be approximated through the use of an optimization-by-simulation approach. The feedback control law is restricted to belong to an a priori fixed class of feedback control laws depending on a (small) finite set of parameters. Under some general conditions developed in this paper, infinitesimal perturbation analysis (IPA) can be used to estimate the gradient of the objective function with respect to these parameters for finite horizon simulation and the consistency of the IPA estimators, as the simulation length goes to infinity, is assured. Also, the parameters can be optimized through a stochastic approximation (SA) algorithm combined with IPA. We prove that in this context, under appropriate conditions, such an approach converges towards the optimum.

[1]  J. Dieudonne Foundations of Modern Analysis , 1969 .

[2]  A. Gut On the moments of some first passage times for sums of dependent random variables , 1974 .

[3]  R. Rishel Dynamic Programming and Minimum Principles for Systems with Jump Markov Disturbances , 1975 .

[4]  Michael P. Polis,et al.  Reducing energy consumption through trajectory optimization for a metro network , 1975 .

[5]  A. Haurie On some properties of the characteristic function and the core of a multistage game of coalitions , 1975 .

[6]  Alain Haurie,et al.  On Existence of Overtaking Optimal Trajectories Over an Infinite Time Horizon , 1976, Math. Oper. Res..

[7]  Harold J. Kushner,et al.  wchastic. approximation methods for constrained and unconstrained systems , 1978 .

[8]  G. Pflug Stochastic Approximation Methods for Constrained and Unconstrained Systems - Kushner, HJ.; Clark, D.S. , 1980 .

[9]  J.-C. Bernard,et al.  Modele D’Optimisation Des Courbes D’Alerte Pour La Gestion De Reservoirs En Vue De L’Irrigation Et De La Production D’Electricite , 1981 .

[10]  John N. Tsitsiklis,et al.  Convexity and characterization of optimal policies in a dynamic routing problem , 1984 .

[11]  Stanley B. Gershwin,et al.  An algorithm for the computer control of a flexible manufacturing system , 1983 .

[12]  Mark H. A. Davis Piecewise‐Deterministic Markov Processes: A General Class of Non‐Diffusion Stochastic Models , 1984 .

[13]  Stanley B. Gershwin,et al.  Short term production scheduling of an automated manufacturing facility , 1984, The 23rd IEEE Conference on Decision and Control.

[14]  R. Akella,et al.  Optimal control of production rate in a failure prone manufacturing system , 1985, 1985 24th IEEE Conference on Decision and Control.

[15]  D. Verms Optimal control of piecewise deterministic markov process , 1985 .

[16]  Ram Akella,et al.  Optimal control of production rate in a failure prone manufacturing system , 1985 .

[17]  Peter W. Glynn,et al.  Stochastic approximation for Monte Carlo optimization , 1986, WSC '86.

[18]  Xi-Ren Cao Sensitivity estimates based on one realization of a stochastic system , 1987 .

[19]  Yu-Chi Ho,et al.  Performance evaluation and perturbation analysis of discrete event dynamic systems , 1987 .

[20]  A. Gut Stopped Random Walks , 1987 .

[21]  A. Sharifnia,et al.  Production control of a manufacturing system with multiple machine states , 1988 .

[22]  Oded Maimon,et al.  Dynamic Scheduling and Routing for Flexible Manufacturing Systems that Have Unreliable Machines , 1988, Oper. Res..

[23]  A. Haurie,et al.  Planning production and preventive maintenance in a flexible manufacturing system: a stochastic control approach , 1988, Proceedings of the 27th IEEE Conference on Decision and Control.

[24]  Panganamala Ramana Kumar,et al.  Optimality of Zero-Inventory Policies for Unreliable Manufacturing Systems , 1988, Oper. Res..

[25]  A. Haurie,et al.  Optimality conditions for continuous time systems with controlled jump Markov disturbances: application to an FMS planning problem , 1988 .

[26]  El-Kébir Boukas,et al.  Manufacturing flow control and preventing maintenance: a stochastic control approach , 1988 .

[27]  P. H. Algoet,et al.  Flow balance equations for the steady-state distribution of a flexible manufacturing system , 1989 .

[28]  Ronald W. Wolff,et al.  Stochastic Modeling and the Theory of Queues , 1989 .

[29]  Juan Ye Optimal control of piecewise deterministic Markov processes. , 1990 .

[30]  M. Fu Convergence of a stochastic approximation algorithm for the GI/G/1 queue using infinitesimal perturbation analysis , 1990 .

[31]  P. L’Ecuyer,et al.  A Unified View of the IPA, SF, and LR Gradient Estimation Techniques , 1990 .

[32]  Paul Glasserman,et al.  Gradient Estimation Via Perturbation Analysis , 1990 .

[33]  Christian van Delft,et al.  Turnpike properties for a class of piecewise deterministic systems arising in manufacturing flow control , 1989, Ann. Oper. Res..

[34]  Pierre L'Ecuyer,et al.  An overview of derivative estimation , 1991, 1991 Winter Simulation Conference Proceedings..

[35]  R. Malhamé,et al.  A renewal theoretic analysis of a class of manufacturing systems , 1991 .

[36]  George Liberopoulos,et al.  Perturbation Analysis for the Design of Flexible Manufacturing System Flow Controllers , 1992, Oper. Res..

[37]  Houmin Yan,et al.  Approximating optimal threshold values for unreliable manufacturing systems via stochastic optimization , 1992, [1992] Proceedings of the 31st IEEE Conference on Decision and Control.

[38]  P. Glynn,et al.  Stochastic Optimization by Simulation: Convergence Proofs for the GI/G/1 Queue in Steady-State , 1994 .

[39]  E. Chong,et al.  Optimization of queues using an infinitesimal perturbation analysis-based stochastic algorithm with general update times , 1993 .

[40]  E. Chong,et al.  Stochastic optimization of regenerative systems using infinitesimal perturbation analysis , 1994, IEEE Trans. Autom. Control..

[41]  P. Glynn,et al.  Stochastic optimization by simulation: numerical experiments with the M / M /1 queue in steady-state , 1994 .