Dynamic job-shop scheduling using reinforcement learning agents

Static and dynamic scheduling methods have attracted a lot of attention in recent years. Among these, dynamic scheduling techniques handle scheduling problems where the scheduler does not possess detailed information about the jobs, which may arrive at the shop at any time. In this paper, an intelligent agent based dynamic scheduling system is proposed. It consists of two independent components: the agent and the simulated environment. The agent selects the most appropriate priority rule according to the shop conditions in real time, while simulated environment performs scheduling activities using the rule selected by the agent. The agent is trained by an improved reinforcement learning algorithm through the learning stage and then it successively makes decisions to schedule the operations.

[1]  T. Ross Fuzzy Logic with Engineering Applications , 1994 .

[2]  Joseph J. Talavage,et al.  A transient-based real-time scheduling algorithm in FMS , 1991 .

[3]  Rodney A. Brooks,et al.  Learning to Coordinate Behaviors , 1990, AAAI.

[4]  Barbara Hayes Roth Architectural foundations for real-time performance in intelligent agents , 1990 .

[5]  Long Ji Lin,et al.  Reinforcement Learning of Non-Markov Decision Processes , 1995, Artif. Intell..

[6]  Luc Steels,et al.  The artificial life roots of artificial intelligence , 1993 .

[7]  Shinichi Nakasuka,et al.  Dynamic scheduling system utilizing machine learning as a knowledge acquisition tool , 1992 .

[8]  Yohanan Arzi,et al.  On-line scheduling in a multi-cell flexible manufacturing system , 1995 .

[9]  Arne Thesen,et al.  SEMI-MARKOV DECISION MODELS FOR REAL-TIME SCHEDULING , 1991 .

[10]  Satinder Singh Transfer of Learning by Composing Solutions of Elemental Sequential Tasks , 1992, Mach. Learn..

[11]  Haruki Matsuura,et al.  Sequencing, dispatching and switching in a dynamic manufacturing environment , 1993 .

[12]  Nicholas R. Jennings,et al.  Intelligent agents: theory and practice , 1995, The Knowledge Engineering Review.

[13]  Chris Watkins,et al.  Learning from delayed rewards , 1989 .

[14]  Feng-Chang Chang A knowledge-based real-time decision support system for job shop scheduling at the shop floor level / , 1985 .

[15]  Sridhar Mahadevan,et al.  Automatic Programming of Behavior-Based Robots Using Reinforcement Learning , 1991, Artif. Intell..

[16]  K. T. Yeo,et al.  An expert neural network system for dynamic job shop scheduling , 1994 .

[17]  Michael J. Shaw,et al.  Intelligent Scheduling with Machine Learning Capabilities: The Induction of Scheduling Knowledge§ , 1992 .

[18]  Nanua Singh Systems Approach to Computer-Integrated Design and Manufacturing , 1995 .

[19]  Andrew G. Barto,et al.  Learning to Act Using Real-Time Dynamic Programming , 1995, Artif. Intell..

[20]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[21]  Wei Zhang,et al.  A Reinforcement Learning Approach to job-shop Scheduling , 1995, IJCAI.

[22]  Gerald Tesauro,et al.  Practical Issues in Temporal Difference Learning , 1992, Mach. Learn..

[23]  A. J. Clewett,et al.  Introduction to sequencing and scheduling , 1974 .

[24]  Marco Colombetti,et al.  Robot Shaping: Developing Autonomous Agents Through Learning , 1994, Artif. Intell..

[25]  Erik D. Goodman,et al.  A Genetic Algorithm Approach to Dynamic Job Shop Scheduling Problem , 1997, ICGA.

[26]  C. Bierwirth,et al.  Genetic algorithm based scheduling in a dynamic manufacturing environment , 1995, Proceedings of 1995 IEEE International Conference on Evolutionary Computation.

[27]  Rodney A. Brooks,et al.  Intelligence Without Reason , 1991, IJCAI.

[28]  Li Lin,et al.  A dynamic job shop scheduling framework: a backward approach , 1994 .