Dynamic Scheduling in Petroleum Process using Reinforcement Learning

Petroleum industry production systems are highly automatized. In this industry, all functions (e.g., planning, scheduling and maintenance) are automated and in order to remain competitive researchers attempt to design an adaptive control system which optimizes the process, but also able to adapt to rapidly evolving demands at a fixed cost. In this paper, we present a multi-agent approach for the dynamic task scheduling in petroleum industry production system. Agents simultaneously insure effective production scheduling and the continuous improvement of the solution quality by means of reinforcement learning, using the SARSA algorithm. Reinforcement learning allows the agents to adapt, learning the best behaviors for their various roles without reducing the performance or reactivity. To demonstrate the innovation of our approach, we include a computer simulation of our model and the results of experimentation applying our model to an Algerian petroleum refinery.

[1]  Reid G. Smith,et al.  The Contract Net Protocol: High-Level Communication and Control in a Distributed Problem Solver , 1980, IEEE Transactions on Computers.

[2]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[3]  David Naso,et al.  Discrete-event modeling of heterarchical manufacturing control systems , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[4]  Damien Trentesaux,et al.  Use of machine learning for continuous improvement of the real time heterarchical manufacturing control system performances , 2008 .

[5]  László Monostori,et al.  Adaptive algorithms in distributed resource allocation , 2006 .

[6]  Damien Trentesaux,et al.  Les systèmes de pilotage hétérarchiques : innovations réelles ou modèles stériles ? , 2007 .

[7]  Damien Trentesaux,et al.  A multicriteria decision support system for dynamic task allocation in a distributed production activity control structure , 1998, Int. J. Comput. Integr. Manuf..

[8]  MY Zhao,et al.  A Reinforcement Learning-based Approach to Dynamic Job-shop Scheduling , 2005 .

[9]  Patrick Pujo,et al.  Pilotage sans plan prévisionnel ni ordonnancement préalable , 2001 .

[10]  Mitsuo Kawato,et al.  Heterarchical reinforcement-learning model for integration of multiple cortico-striatal loops: fMRI examination in stimulus-action-reward association learning , 2006, Neural Networks.

[11]  Richard S. Sutton,et al.  Reinforcement learning with replacing eligibility traces , 2004, Machine Learning.

[12]  Damien Trentesaux,et al.  Self-organization in distributed manufacturing control: state-of-the-art and future trends , 2002, IEEE International Conference on Systems, Man and Cybernetics.

[13]  J. Christopher Beck,et al.  A General Framework for Scheduling in a Stochastic Environment , 2007, IJCAI.

[14]  N. A. Duffie Heterarchical control of highly distributed manufacturing systems , 1996 .

[15]  Mehmet Emin Aydin,et al.  Dynamic job-shop scheduling using reinforcement learning agents , 2000, Robotics Auton. Syst..

[16]  Vittaldas V. Prabhu Stable fault adaptation in distributed control of heterarchical manufacturing job shops , 2003, IEEE Trans. Robotics Autom..

[17]  Peter Norvig,et al.  Artificial intelligence - a modern approach: the intelligent agent book , 1995, Prentice Hall series in artificial intelligence.