A Reinforcement Learning Approach to Robust Scheduling of Semiconductor Manufacturing Facilities

As semiconductor manufacturers, recently, have focused on producing multichip products (MCPs), scheduling semiconductor manufacturing operations become complicated due to the constraints related to reentrant production flows, sequence-dependent setups, and alternative machines. At the same time, the scheduling problems need to be solved frequently to effectively manage the variabilities in production requirements, available machines, and initial setup status. To minimize the makespan for an MCP scheduling problem, we propose a setup change scheduling method using reinforcement learning (RL) in which each agent determines setup decisions in a decentralized manner and learns a centralized policy by sharing a neural network among the agents to deal with the changes in the number of machines. Furthermore, novel definitions of state, action, and reward are proposed to address the variabilities in production requirements and initial setup status. Numerical experiments demonstrate that the proposed approach outperforms the rule-based, metaheuristic, and other RL methods in terms of the makespan while incurring shorter computation time than the metaheuristics considered. Note to Practitioners—This article studies a scheduling problem for die attach and wire bonding stages of a semiconductor packaging line. Due to the variabilities in production requirements, the number of available machines, and initial setup status, it is challenging for a scheduler to produce high-quality schedules within a specific time limit using existing approaches. In this article, a new scheduling method using reinforcement learning is proposed to enhance the robustness against the variabilities while achieving performance improvements. To verify the robustness of the proposed method, neural networks (NNs) trained on small-scale scheduling problems are used to solve large-scale scheduling problems. Experimental results show that the proposed method outperforms the existing approaches while requiring a short computation time. Furthermore, the trained NN performs well in solving unseen real-world scale problems even under stochastic processing time, suggesting the viability of the proposed method for real-world semiconductor packaging lines.