Dynamic scheduling for multi-site companies: a decisional approach based on reinforcement multi-agent learning

In recent years, most companies have resorted to multi-site or supply-chain organization in order to improve their competitiveness and adapt to existing real conditions. In this article, a model for adaptive scheduling in multi-site companies is proposed. To do this, a multi-agent approach is adopted in which intelligent agents have reactive learning capabilities based on reinforcement learning. This reactive learning technique allows the agents to make accurate short-term decisions and to adapt these decisions to environmental fluctuations. The proposed model is implemented on a 3-tier architecture that ensures the security of the data exchanged between the various company sites. The proposed approach is compared to a genetic algorithm and a mixed integer linear program algorithm to prove its feasibility and especially, its reactivity. Experimentations on a real case study demonstrate the applicability and the effectiveness of the model in terms of both optimality and reactivity.

[1]  Ram Narasimhan,et al.  Decision models in global supply chain management , 2004 .

[2]  Jacques Ferber,et al.  A meta-model for the analysis and design of organizations in multi-agent systems , 1998, Proceedings International Conference on Multi Agent Systems (Cat. No.98EX160).

[3]  Luca Maria Gambardella,et al.  Effective Neighborhood Functions for the Flexible Job Shop Problem , 1998 .

[4]  Damien Trentesaux,et al.  Heterarchical production control in manufacturing systems using the potential fields concept , 2012, J. Intell. Manuf..

[5]  Damien Trentesaux,et al.  Self-organization in distributed manufacturing control: state-of-the-art and future trends , 2002, IEEE International Conference on Systems, Man and Cybernetics.

[6]  Nobuhiro Sugimura,et al.  Multi agent architecture for dynamic incremental process planning in the flexible manufacturing system , 2010, J. Intell. Manuf..

[7]  Jayashankar M. Swaminathan,et al.  Modeling Supply Chain Dynamics: A Multiagent Approach , 1998 .

[8]  Paolo Brandimarte,et al.  Routing and scheduling in a flexible job shop by tabu search , 1993, Ann. Oper. Res..

[9]  Damien Trentesaux,et al.  Distributed control of production systems , 2009, Eng. Appl. Artif. Intell..

[10]  Luc Bongaerts,et al.  Reference architecture for holonic manufacturing systems: PROSA , 1998 .

[11]  Nuno Silva,et al.  A Holonic Manufacturing System implementation , 2001 .

[12]  Mitsuo Gen,et al.  A hybrid of genetic algorithm and bottleneck shifting for multiobjective flexible job shop scheduling problems , 2007, Comput. Ind. Eng..

[13]  Christos D. Tarantilis,et al.  Topics in Real-time Supply Chain Management , 2008, Comput. Oper. Res..

[14]  Damien Trentesaux,et al.  Distributed artificial intelligence for FMS scheduling, control and design support , 2000, J. Intell. Manuf..

[15]  Henry Mintzberg,et al.  Structure in 5's: A Synthesis of the Research on Organization Design , 1980, Management Science.

[16]  Geoffrey Vilcot Algorithmes approchés pour des problèmes d'ordonnancement multicritères de type job shop flexible et job shop multiressource , 2007 .

[17]  Jung-Fa Tsai,et al.  An optimization approach for supply chain management models with quantity discount policy , 2007, Eur. J. Oper. Res..

[18]  Nicolas Soulié,et al.  Organisational and spatial determinants of the multi-unit firm: Evidence from the French industry , 2007 .

[19]  M. J. Rosenblatt,et al.  A generalized quantity discount pricing model to increase supplier's profits , 1986 .

[20]  W. McCulloch The heterarchy of values determined by the topology of nervous nets. , 1945, The Bulletin of mathematical biophysics.

[21]  Nicolas Soulié,et al.  Cahiers du GRES Organisational and spatial determinants of the multi- unit firm: Evidence from the French industry , 2007 .

[22]  Christos D. Tarantilis,et al.  Exact, Heuristic and Meta-heuristic Algorithms for Solving Shop Scheduling Problems , 2008, Metaheuristics for Scheduling in Industrial and Manufacturing Applications.

[23]  Botond Kádár,et al.  Adaptation and Learning in Distributed Production Control , 2004 .

[24]  Mahesan Niranjan,et al.  On-line Q-learning using connectionist systems , 1994 .

[25]  Keiki Takadama,et al.  Lessons Learned from Comparison Between Q-learning and Sarsa Agents in Bargaining Game , 2004 .

[26]  Tabitha L. James,et al.  A hybrid grouping genetic algorithm for the cell formation problem , 2007, Comput. Oper. Res..

[27]  P. C. Yang,et al.  Production , Manufacturing and Logistics Pricing strategy for deteriorating items using quantity discount when demand is price sensitive , 2004 .

[28]  Pierre Borne,et al.  Approach by localization and multiobjective evolutionary optimization for flexible job-shop scheduling problems , 2002, IEEE Trans. Syst. Man Cybern. Part C.

[29]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[30]  Damien Trentesaux,et al.  Use of machine learning for continuous improvement of the real time heterarchical manufacturing control system performances , 2008 .

[31]  Richard T. Wong,et al.  General models for the supplier's all-unit quantity discount policy , 1993 .

[32]  Wesley W. Chu,et al.  Solving the Generalized Job Shop Scheduling Problem via Temporal Constraint Propagation , 1990, KBCS.

[33]  Nidhal Rezg,et al.  Geometric approach and taboo search for scheduling flexible manufacturing systems , 2001, IEEE Trans. Robotics Autom..

[34]  James P. Monahan A Quantity Discount Pricing Model to Increase Vendor Profits , 1984 .

[35]  Fariborz Jolai,et al.  Mathematical modeling and heuristic approaches to flexible job shop scheduling problems , 2007, J. Intell. Manuf..

[36]  Paulo Leitão,et al.  A Holonic Approach to Dynamic Manufacturing Scheduling , 2006, BASYS.

[37]  David E. Goldberg,et al.  AllelesLociand the Traveling Salesman Problem , 1985, ICGA.

[38]  Christopher S. Tang The impact of uncertainty on a production line , 1990 .

[39]  Mitsuo Gen,et al.  Multistage-Based Genetic Algorithm for Flexible Job-Shop Scheduling Problem , 2009 .

[40]  Sotiris Makris,et al.  Supply chain modeling and control for producing highly customized products , 2008 .

[41]  Stéphane Dauzère-Pérès,et al.  Modelling and solving a practical flexible job-shop scheduling problem with blocking constraints , 2011 .

[42]  Peter Norvig,et al.  Artificial intelligence - a modern approach: the intelligent agent book , 1995, Prentice Hall series in artificial intelligence.

[43]  Didier Gourc,et al.  A supply chain performance analysis of a pull inspired supply strategy faced to demand uncertainties , 2012, J. Intell. Manuf..

[44]  Damien Trentesaux,et al.  Dynamic scheduling of maintenance tasks in the petroleum industry: A reinforcement approach , 2009, Eng. Appl. Artif. Intell..

[45]  Damien Trentesaux,et al.  A stigmergic approach for dynamic routing of active products in FMS , 2009, Comput. Ind..

[46]  Chris Watkins,et al.  Learning from delayed rewards , 1989 .

[47]  Hoong Chuin Lau,et al.  Real-time supply chain control via multi-agent adjustable autonomy , 2008, Comput. Oper. Res..

[48]  Aleksandar Lazinica,et al.  Bionic assembly system: new concept of self-organising multirobot system , 2007, Int. J. Autom. Control..

[49]  Kanji Ueda,et al.  An emergent modelling method for dynamic scheduling , 1998, J. Intell. Manuf..

[50]  D. Trentesaux,et al.  An approach for temporal myopia reduction in Heterarchical Control Architectures , 2011, 2011 IEEE International Symposium on Industrial Electronics.

[51]  R. E. Miles,et al.  Managing 21st century network organizations , 1992 .

[52]  Stéphane Dauzère-Pérès,et al.  An integrated approach for modeling and solving the general multiprocessor job-shop scheduling problem using tabu search , 1997, Ann. Oper. Res..

[53]  Vittaldas V. Prabhu Stable fault adaptation in distributed control of heterarchical manufacturing job shops , 2003, IEEE Trans. Robotics Autom..

[54]  David L. Woodruff,et al.  Introduction to Computational Optimization Models for Production Planning in a Supply Chain , 2003 .

[55]  Erfu Yang,et al.  Fuzzy Policy Reinforcement Learning in Cooperative Multi-robot Systems , 2007, J. Intell. Robotic Syst..

[56]  David E. Goldberg,et al.  Alleles, loci and the traveling salesman problem , 1985 .

[57]  Mitsuo Kawato,et al.  Heterarchical reinforcement-learning model for integration of multiple cortico-striatal loops: fMRI examination in stimulus-action-reward association learning , 2006, Neural Networks.

[58]  F. Pezzella,et al.  A genetic algorithm for the Flexible Job-shop Scheduling Problem , 2008, Comput. Oper. Res..

[59]  Jürgen Sauer,et al.  Towards agent-based multi-site scheduling , 2000, PuK.

[60]  Mustapha Ouhimmou,et al.  Furniture supply chain tactical planning optimization using a time decomposition approach , 2008, Eur. J. Oper. Res..

[61]  Paulo Leitão,et al.  A holonic approach to dynamic manufacturing scheduling , 2008 .

[62]  Optimization of fresh–food supply chains in uncertain environments: an application to the meat-refrigeration process , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[63]  Patrick Pujo,et al.  Pull control for job shop: holonic manufacturing system approach using multicriteria decision-making , 2012, J. Intell. Manuf..

[64]  William L. Maxwell,et al.  Theory of scheduling , 1967 .

[65]  Peter Brucker,et al.  Job-shop scheduling with multi-purpose machines , 1991, Computing.