Improving iterative repair strategies for scheduling with the SVM

The resource constraint project scheduling problem (RCPSP) is an NP-hard benchmark problem in scheduling which takes into account the limitation of resources' availabilities in real life production processes and subsumes open-shop, job-shop, and flow-shop scheduling as special cases. We present here an application of machine learning to adapt simple greedy strategies for the RCPSP. Iterative repair steps are applied to an initial schedule which neglects resource constraints. The rout-algorithm of reinforcement learning is used to learn an appropriate value function which guides the search. We propose three different ways to define the value function and we use the support vector machine (SVM) for its approximation. The specific properties of the SVM allow to reduce the size of the training set and SVM shows very good generalization behavior also after short training. We compare the learned strategies to the initial greedy strategy for different benchmark instances of the RCPSP.

[1]  Andrew G. Barto,et al.  Improving Elevator Performance Using Reinforcement Learning , 1995, NIPS.

[2]  V. Jorge Leon,et al.  Strength and adaptability of problem-space based neighborhoods for resource-constrained scheduling , 1995 .

[3]  Hartmut Schmeck,et al.  Ant colony optimization for resource-constrained project scheduling , 2000, IEEE Trans. Evol. Comput..

[4]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[5]  Andreas Schirmer,et al.  Case‐based reasoning and improved adaptive search for project scheduling , 2000 .

[6]  Rainer Kolisch,et al.  PSPLIB - A project scheduling problem library: OR Software - ORSEP Operations Research Software Exchange Program , 1997 .

[7]  Andrew W. Moore,et al.  Learning evaluation functions for global optimization , 1998 .

[8]  Roman Słowiński,et al.  Advances in project scheduling , 1989 .

[9]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[10]  Barbara Hammer,et al.  A Note on the Universal Approximation Capability of Support Vector Machines , 2003, Neural Processing Letters.

[11]  Erik Demeulemeester,et al.  New Benchmark Results for the Resource-Constrained Project Scheduling Problem , 1997 .

[12]  Silvano Martello,et al.  Meta-Heuristics: Advances and Trends in Local Search Paradigms for Optimization , 2012 .

[13]  Rainer Kolisch,et al.  Semi-active, active, and non-delay schedules for the resource-constrained project scheduling problem , 1995 .

[14]  R. Shah,et al.  Least Squares Support Vector Machines , 2022 .

[15]  A. Richard Newton,et al.  Learning as applied to stochastic optimization for standard cell placement , 1998, Proceedings International Conference on Computer Design. VLSI in Computers and Processors (Cat. No.98CB36273).

[16]  Martin A. Riedmiller,et al.  A Neural Reinforcement Learning Approach to Learn Local Dispatching Policies in Production Scheduling , 1999, IJCAI.

[17]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[18]  Andrew W. Moore,et al.  Learning Evaluation Functions for Large Acyclic Domains , 1996, ICML.

[19]  Andrew W. Moore,et al.  Stochastic production scheduling to meet demand forecasts , 1998, Proceedings of the 37th IEEE Conference on Decision and Control (Cat. No.98CH36171).

[20]  Robert H. Storer,et al.  Problem space search algorithms for resource-constrained project scheduling , 1997, Ann. Oper. Res..

[21]  Kagan Tumer,et al.  Using Collective Intelligence to Route Internet Traffic , 1998, NIPS.

[22]  Peter Brucker,et al.  A branch and bound algorithm for the resource-constrained project scheduling problem , 1998, Eur. J. Oper. Res..

[23]  Wei Zhang,et al.  A Reinforcement Learning Approach to job-shop Scheduling , 1995, IJCAI.

[24]  R. Bellman.  Dynamic programming. , 1957, Science.

[25]  Ramón Alvarez-Valdés Olaguíbel,et al.  Chapter 5 – HEURISTIC ALGORITHMS FOR RESOURCE-CONSTRAINED PROJECT SCHEDULING: A REVIEW AND AN EMPIRICAL ANALYSIS , 1989 .

[26]  Rainer Kolisch,et al.  Adaptive search for solving hard project scheduling problems , 1996 .

[27]  Helmut Mausser,et al.  Exploiting Block Structure to Improve Resource-Constrained Project Schedules , 1996 .

[28]  Philip M. Wolfe,et al.  Multiproject Scheduling with Limited Resources: A Zero-One Programming Approach , 1969 .

[29]  Kee-Eung Kim,et al.  Statistical Machine Learning for Large-Scale Optimization , 2000 .

[30]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[31]  Colin E. Bell,et al.  A new heuristic solution method in resource‐constrained project scheduling , 1991 .

[32]  Andrew G. Barto,et al.  Building a Basic Block Instruction Scheduler with Reinforcement Learning and Rollouts , 2002, Machine Learning.

[33]  Bruce Pollack-Johnson,et al.  Hybrid structures and improving forecasting and scheduling in project management , 1995 .

[34]  A. Alan B. Pritsker,et al.  Multiproject Scheduling with Limited Resources , 1968 .

[35]  Jan Karel Lenstra,et al.  Scheduling subject to resource constraints: classification and complexity , 1983, Discret. Appl. Math..

[36]  James H. Patterson,et al.  Scheduling a Project Under Multiple Resource Constraints: A Zero-One Programming Approach , 1976 .

[37]  Yeong-Dae Kim,et al.  Search Heuristics for Resource Constrained Project Scheduling , 1996 .

[38]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[39]  Michael Luck,et al.  Proceedings of the Third International Conference on Multi-Agent Systems , 1998 .

[40]  P. Brucker,et al.  Tabu Search Algorithms and Lower Bounds for the Resource-Constrained Project Scheduling Problem , 1999 .

[41]  Rema Padman,et al.  An integrated survey of project scheduling , 1997 .

[42]  Rainer Kolisch,et al.  Efficient priority rules for the resource-constrained project scheduling problem , 1996 .

[43]  Jakob Stoustrup,et al.  Fault detection for nonlinear systems - a standard problem approach , 1998, Proceedings of the 37th IEEE Conference on Decision and Control (Cat. No.98CH36171).

[44]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[45]  Peter Brucker,et al.  Lower bounds for resource-constrained project scheduling problems , 2003, Eur. J. Oper. Res..

[46]  Albert Battersby,et al.  Advances in Critical Path Methods , 1966 .

[47]  Andrew W. Moore,et al.  Value Function Based Production Scheduling , 1998, ICML.

[48]  Artur Merke,et al.  A Necessary Condition of Convergence for Reinforcement Learning with Function Approximation , 2002, ICML.

[49]  Wilfried Brauer,et al.  Multi-machine scheduling-a multi-agent learning approach , 1998, Proceedings International Conference on Multi Agent Systems (Cat. No.98EX160).

[50]  Thorsten Joachims,et al.  Optimizing search engines using clickthrough data , 2002, KDD.

[51]  Roberto Basili,et al.  Learning to Classify Text Using Support Vector Machines: Methods, Theory, and Algorithms by Thorsten Joachims , 2003, Comput. Linguistics.

[52]  V. Maniezzo,et al.  An Exact Algorithm for the Resource-Constrained Project Scheduling Problem Based on a New Mathematical Formulation , 1998 .

[53]  Sönke Hartmann,et al.  A competitive genetic algorithm for resource-constrained project scheduling , 1998 .

[54]  Elliott N. Weiss,et al.  Local search techniques for the generalized resource constrained project scheduling problem , 1993 .

[55]  Richard S. Sutton,et al.  Learning Instance-Independent Value Functions to Enhance Local Search , 1998, NIPS.