Dynamic programming in a heuristically confined state space: a stochastic resource-constrained project scheduling application

Abstract The resource-constrained project scheduling problem (RCPSP) is a significant challenge in highly regulated industries, such as pharmaceuticals and agrochemicals, where a large number of candidate new products must undergo a set of tests for certification. We propose a novel way of addressing the uncertainties in the RCPSP including the uncertainties in task durations and costs, as well as uncertainties in the results of tasks (success or failure) by using a discrete time Markov chain, which enables us to model probabilistic correlation of the uncertain parameters. The resulting stochastic optimization problem can be solved by using dynamic programming, but the computational load renders this impractical. Instead, we develop a new way to combine heuristic solutions through dynamic programming in the state space that the heuristics generate. The proposed approach is tested on a simplified version of RCPSP that has a fairly complicated stochastic nature, with 1,214,693,756 possible parameter realizations (scenarios), and involves five projects and 17 tasks. As a result, an on-line policy is obtained, which can use the information states in on-line decision making and improve the heuristics rather than a fixed solution obtained by the previous mathematical programming (MILP) problem formulations.

[1]  Jay H. Lee,et al.  An algorithmic framework for improving heuristic solutions: Part II. A new version of the stochastic traveling salesman problem , 2004, Comput. Chem. Eng..

[2]  Vipul Jain,et al.  Resource-Constrained Scheduling of Tests in New Product Development , 1999 .

[3]  Jan Karel Lenstra,et al.  Scheduling subject to resource constraints: classification and complexity , 1983, Discret. Appl. Math..

[4]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[5]  Gintaras V. Reklaitis,et al.  A simulation‐optimization framework for research and development pipeline management , 2001 .

[6]  Gary Blau,et al.  Risk management in the development of new products in highly regulated industries , 2000 .

[7]  Eswaran Subrahmanian,et al.  Design and planning under uncertainty: issues on problem formulation and solution , 2003, Comput. Chem. Eng..

[8]  Ignacio E. Grossmann,et al.  Optimization Models for the Scheduling of Testing Tasks in New Product Development , 1996 .

[9]  Gintaras V. Reklaitis,et al.  Simulation-optimization framework for stochastic optimization of R&D pipeline management , 2003 .

[10]  Jay H. Lee,et al.  Simulation based approach for improving heuristics in stochastic resource-constrained project scheduling problem , 2003 .

[11]  I. Grossmann,et al.  Simultaneous Planning for New Product Development and Batch Manufacturing Facilities , 2001 .

[12]  Costas D. Maranas,et al.  Real Options Based Analysis of Optimal Pharmaceutical Research and Development Portfolios , 2002 .

[13]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[14]  Jay H. Lee,et al.  An algorithmic framework for improving heuristic solutions: Part I. A deterministic discount coupon traveling salesman problem , 2004, Comput. Chem. Eng..

[15]  Gintaras V. Reklaitis,et al.  A simulation—optimization framework for addressing combinatorial and stochastic aspects of an R&D pipeline management problem , 2000 .