Adaptive Critic Design of Control Policies for A Multi-Echelon Inventory System

An abstract of the dissertation of Stephen Shervais for the Doctor of Philosophy in Systems Science presented October 6, 2000. Title: Adaptive Critic Design of Control Policies For A Multi-Echelon Inventory System A common problem in business is the determination of inventory and transportation policies for a physical distribution system within a changing business environment. This dissertation addresses the process of selecting an optimal set of policies for a multi-product, multi-echelon, multi-modal physical distribution system in a nonstationary environment. The problem is highly multi-dimensional, even with a small system, and the fitness surface is quite often discontinuous, with low penalty and high penalty regions separated by no more than a single transport unit. A controller design process is presented that reliably improves on the performance of the typical fixedpolicy controllers. The design process has two basic stages. First, a Genetic Algorithm (GA) is used to perform a global search (in a static environment) to find a good initial policy to be used as a starting point by the next stage. Second, an approximate dynamic programming method, implemented by an adaptive critic method known as Dual Heuristic Programming (DHP), is used to perform local optimization and fitness-terrainfollowing in a changing environment, to yield (approximately) optimal control policies. The design process includes use of training data embodying 1/f noise. Performance of the resulting controllers for the defined context was compared with that of fixed-policy controllers -in this case, a fixed policy developed using the well-known Linear Programming (LP) method, and in addition, a fixed policy developed via a GA approach. Significantly, even the worst controller developed via the proposed method substantially outperformed both the LP and GA fixed policies. On a real-world generalization test set, the average total cost score for neural-adaptive controllers was 464, while fixed-policy controllers based on LP optimization scored costs of 2891, and GAbased fixed policy controllers scored 3477. In addition, I demonstrate the effectiveness of off-optimal, GA-developed I/O pairs as the basis for training a neural-network model of the plant (needed for the DHP process). Further, I speculate on the use of a GA as a way of testing candidate business rules for structural errors.