Applying hybrid Monte Carlo Tree Search methods to Risk-Aware Project Scheduling Problem

Abstract In this paper we investigate an application of hybrid Monte Carlo Tree Search (MCTS) based algorithms to solving dynamic decision making problems. We employ UCT (the most popular MCTS approach) in combination with well-known Resource Constrained Project Scheduling Problem (RCPSP) and Stochastic Resource Constrained Project Scheduling Problem (SRCPSP) solvers to devise strategies for a generic and highly dynamic version of RCPSP, which we call Risk-Aware Project Scheduling Problem (RAPSP). We compare these strategies’ performance with results of both pure MCTS approach and non-MCTS solvers for projects of varied characteristics. We reach a conclusion that proposed hybrid simulation-heuristic methods are promising approaches to dynamic decision making problems, RAPSP in particular. Consequently, we argue that more research effort should be directed to applications of MCTS algorithm outside the domain of game-playing, with which it is commonly associated. At the same time, to the best of our knowledge, this paper is the first attempt at defining generalized SRCPSP model encompassing arbitrary risks and risk response / mitigation strategies as an optimization problem and applying Computational Intelligence methods to build fully-automated decision making systems. We strongly believe it to be a research direction worth further investigation, combining project scheduling, risk management and metaheuristic optimization techniques into a well-defined platform allowing direct comparisons of different strategies.

[1]  Maciej Swiechowski,et al.  Simulation-based approach to Vehicle Routing Problem with traffic jams , 2016, 2016 IEEE Symposium Series on Computational Intelligence (SSCI).

[2]  Shimpei Matsumoto,et al.  Worker’s knowledge evaluation with single-player Monte Carlo tree search for a practical reentrant scheduling problem , 2016, Artificial Life and Robotics.

[3]  Ronald L. Graham,et al.  Bounds for certain multiprocessing anomalies , 1966 .

[4]  Jonathan Schaeffer.  The History Heuristic , 1983, J. Int. Comput. Games Assoc..

[5]  Yew-Soon Ong,et al.  Specialization of a UCT-Based General Game Playing Program to Single-Player Games , 2016, IEEE Transactions on Computational Intelligence and AI in Games.

[6]  Maciej Swiechowski,et al.  Self-Adaptation of Playing Strategies in General Game Playing , 2014, IEEE Transactions on Computational Intelligence and AI in Games.

[7]  W. Duncan.  A GUIDE TO THE PROJECT MANAGEMENT BODY OF KNOWLEDGE , 1996 .

[8]  Tyson R. Browning,et al.  Scheduling multiple, resource-constrained, iterative, product development projects with genetic algorithms , 2017, Comput. Ind. Eng..

[9]  John R. Schuyler,et al.  Risk and Decision Analysis in Projects , 2001 .

[10]  A. Alan B. Pritsker,et al.  GERT - Graphical Evaluation and Review Technique , 1966 .

[11]  Maciej Swiechowski,et al.  Generic Heuristic Approach to General Game Playing , 2012, SOFSEM.

[12]  Jacek Mandziuk,et al.  An Automatically Generated Evaluation Function in General Game Playing , 2014, IEEE Transactions on Computational Intelligence and AI in Games.

[13]  Celso C. Ribeiro,et al.  Greedy Randomized Adaptive Search Procedures , 2003, Handbook of Metaheuristics.

[14]  Sönke Hartmann,et al.  A competitive genetic algorithm for resource-constrained project scheduling , 1998 .

[15]  Erik Demeulemeester,et al.  Project scheduling : a research handbook , 2002 .

[16]  Jacek Mandziuk,et al.  Mixed Strategy Extraction from UCT Tree in Security Games , 2016, ECAI.

[17]  Damien Pellier,et al.  An UCT Approach for Anytime Agent-Based Planning , 2010, PAAMS.

[18]  Rainer Kolisch,et al.  PSPLIB - A project scheduling problem library: OR Software - ORSEP Operations Research Software Exchange Program , 1997 .

[19]  N. Agin.  Optimum Seeking with Branch and Bound , 1966 .

[20]  Francisco Ballestín,et al.  Justification and RCPSP: A technique that pays , 2005, Eur. J. Oper. Res..

[21]  Saïd Salhi,et al.  A Tabu Search Approach for the Resource Constrained Project Scheduling Problem , 1998, J. Heuristics.

[22]  Javier Del Ser,et al.  Comparative study of pheromone control heuristics in ACO algorithms for solving RCPSP problems , 2017, Appl. Soft Comput..

[23]  Fayez F. Boctor,et al.  An efficient genetic algorithm to solve the resource-constrained project scheduling problem with transfer times: The single mode case , 2018, Eur. J. Oper. Res..

[24]  Slawomir Zadrozny,et al.  Proactive and reactive risk-aware project scheduling , 2014, 2014 IEEE Symposium on Computational Intelligence for Human-like Intelligence (CIHLI).

[25]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[26]  Ronald R. Yager,et al.  Preferences and Decisions - Models and Applications , 2010, Studies in Fuzziness and Soft Computing.

[27]  Simon M. Lucas,et al.  A Survey of Monte Carlo Tree Search Methods , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[28]  H. Sapolsky.  The Polaris System Development: Bureaucratic and Programmatic Success in Government , 1972 .

[29]  James E. Kelley,et al.  Critical-path planning and scheduling , 1899, IRE-AIEE-ACM '59 (Eastern).

[30]  Mauricio G. C. Resende,et al.  Grasp: An Annotated Bibliography , 2002 .

[31]  Salim Rostami,et al.  New strategies for stochastic resource-constrained project scheduling , 2017, Journal of Scheduling.

[32]  Ender Özcan,et al.  Combining Monte-Carlo and hyper-heuristic methods for the multi-mode resource-constrained multi-project scheduling problem , 2015, Inf. Sci..

[33]  Roel Leus,et al.  Resource‐Constrained Project Scheduling for Timely Project Completion with Stochastic Activity Durations , 2007 .

[34]  Klaus Neumann,et al.  Truncated branch-and-bound, schedule-construction, and schedule-improvement procedures for resource-constrained project scheduling , 2001, OR Spectr..

[35]  Terry Williams,et al.  Why Monte Carlo Simulations of Project Networks Can Mislead , 2004 .

[36]  Jacek Madziuk,et al.  Risk-Aware Project Scheduling for Projects with Varied Risk Levels , 2015, SSCI.

[37]  Jacek Mandziuk,et al.  UCT-Based Approach to Capacitated Vehicle Routing Problem , 2015, ICAISC.

[38]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.