Genetic reinforcement learning for scheduling heterogeneous machines

Concerns the development of a learning-based heuristic for scheduling heterogeneous machines. List scheduling methods are flexible enough to be used for a large class of problems, including the heterogeneous machine problem. However, designing a priority rule requires insight into the characteristics of the problem. We propose the iterative list scheduling, which refines priority rules while generating a number of schedules. We also show that the iterative list scheduling can be formulated as a reinforcement learning problem, defining states and actions. Due to the large number of possible states, reinforcement learning algorithms which use value functions in constructing an optimal policy may not be suitable for scheduling problems. Encoding the policies of reinforcement learning into genetic algorithms leads to the genetic reinforcement learning (GRL), which directly works with the policies rather than the values of states. A GRL-based scheduler, EVIS (Evolutionary Intracell Scheduler), has been applied to problems such as the heterogeneous machine scheduling, the job-shop scheduling, the flow-shop scheduling, and the open-shop scheduling problems. The proposed model of EVIS, which has the linear order of population-fitness convergence, was verified with computer experiments. Even without fine tuning of EVIS, the quality of solutions found by EVIS was comparable to that of problem-tailored heuristics for most of the problem instances.