An improved Q-learning based rescheduling method for flexible job-shops with machine failures

Scheduling of flexible job shop has been researched over several decades and continues to attract the interests of many scholars. But in the real manufacturing system, dynamic events such as machine failures are major issues. In this paper, an improved Q-learning algorithm with double-layer actions is proposed to solve the dynamic flexible job-shop scheduling problem (DFJSP) considering machine failures. The initial scheduling scheme is obtained by Genetic Algorithm (GA), and the rescheduling strategy is acquired by the Agent of the proposed Q-learning based on dispatching rules. The agent of Q-learning is able to select both operations and alternative machines optimally when machine failure occurs. To testify this approach, experiments are designed and performed based on Mk03 problem of FJSP. Results demonstrate that the optimal rescheduling strategy varies in different machine failure status. And compared with adopting a single dispatching rule all the time, the proposed Q-learning can reduce time of delay in a frequent dynamic environment, which shows that agent-based method is suitable for DFJSP.

[1]  Yan Wang,et al.  NSGA-II applied to dynamic flexible job shop scheduling problems with machine breakdown , 2018, Modern Physics Letters B.

[2]  Tao Zhang,et al.  Real-time job shop scheduling based on simulation and Markov decision processes , 2017, 2017 Winter Simulation Conference (WSC).

[3]  Hua Jin,et al.  A novel dynamic scheduling strategy for solving flexible job-shop problems , 2016, J. Ambient Intell. Humaniz. Comput..

[4]  Paolo Brandimarte,et al.  Routing and scheduling in a flexible job shop by tabu search , 1993, Ann. Oper. Res..

[5]  X. P. Yang,et al.  Optimization of Dynamic and Multi-Objective Flexible Job-Shop Scheduling Based on Parallel Hybrid Algorithm , 2018, International Journal of Simulation Modelling.

[6]  Jamal Shahrabi,et al.  A reinforcement learning approach to parameter estimation in dynamic job shop scheduling , 2017, Comput. Ind. Eng..

[7]  Christoph H. Glock,et al.  Machine scheduling problems in production: A tertiary study , 2017, Comput. Ind. Eng..

[8]  Jeffrey W. Herrmann,et al.  Rescheduling Manufacturing Systems: A Framework of Strategies, Policies, and Methods , 2003, J. Sched..

[9]  Lionel Amodeo,et al.  Multi-Objective Scheduling Problems for Re-Entrant Shops , 2012 .

[10]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[11]  Sanja Petrovic,et al.  SURVEY OF DYNAMIC SCHEDULING IN MANUFACTURING SYSTEMS , 2006 .

[12]  Yong Zhou,et al.  Hyper-Heuristic Coevolution of Machine Assignment and Job Sequencing Rules for Multi-Objective Dynamic Flexible Job Shop Scheduling , 2019, IEEE Access.

[13]  Ali Azadeh,et al.  A hybrid computer simulation-artificial neural network algorithm for optimisation of dispatching rule selection in stochastic job shop scheduling problems , 2012 .

[14]  Wu Xiu-li Research on Flexible Job Shop Dynamic Scheduling Problem , 2008 .

[15]  M. A. Adibi,et al.  A clustering-based modified variable neighborhood search algorithm for a dynamic job shop scheduling problem , 2013, The International Journal of Advanced Manufacturing Technology.

[16]  Arun Kumar Sangaiah,et al.  A New Meta-Heuristic Algorithm for Solving the Flexible Dynamic Job-Shop Problem with Parallel Machines , 2019, Symmetry.

[17]  Xinyu Li,et al.  An effective hybrid genetic algorithm and tabu search for flexible job shop scheduling problem , 2016 .

[18]  Yu-Fang Wang,et al.  Adaptive job shop scheduling strategy based on weighted Q-learning algorithm , 2018, Journal of Intelligent Manufacturing.