Service Composition in Cloud Manufacturing: A DQN-Based Approach

Cloud manufacturing is a new service-oriented manufacturing model that integrates distributed manufacturing resources to provide on-demand manufacturing services over the Internet. Service composition that builds larger-granularity, value-added services by combining a number of smaller-granularity services to satisfy consumers’ complex requirements is an important issue in cloud manufacturing. Meta-heuristic algorithms such as genetic algorithm, particle swarm optimization, and ant colony algorithm are frequently employed for addressing service composition issues in cloud manufacturing. However, these algorithms require complex design flows and lack adaptability to dynamic environment. Deep reinforcement learning provides an alternative approach for solving cloud manufacturing service composition issues. This chapter proposes a deep Q-network (DQN) based approach for service composition in cloud manufacturing, which is able to find optimal service composition solutions through repeated training and learning. Results of experiments that take into account changes of service scales and service unavailability reveal the scalability and robustness of the DQN algorithm-based service composition approach.

[1]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[2]  Ren Lei,et al.  Typical characteristics of cloud manufacturing and several key issues of cloud service composition , 2011 .

[3]  Bo Yang,et al.  A dynamic ant-colony genetic algorithm for cloud service composition optimization , 2019, The International Journal of Advanced Manufacturing Technology.

[4]  Sami Kara,et al.  Towards Energy and Resource Efficient Manufacturing: A Processes and Systems Approach , 2012 .

[5]  Jun Zhang,et al.  An autonomy-oriented method for service composition and optimal selection in cloud manufacturing , 2018 .

[6]  Chi-Guhn Lee,et al.  Manufacturing task semantic modeling and description in cloud manufacturing system , 2014 .

[7]  Lei Ren,et al.  Cloud manufacturing: a new manufacturing paradigm , 2014, Enterp. Inf. Syst..

[8]  Yongkui Liu,et al.  Manufacturing Service Management in Cloud Manufacturing: Overview and Future Research Directions , 2015 .

[9]  Philip C. Jackson.  Introduction to Artificial Intelligence , 1985 .

[10]  Alexandre Dolgui,et al.  Scheduling in production, supply chain and Industry 4.0 systems by optimal control: fundamentals, state-of-the-art and applications , 2019, Int. J. Prod. Res..

[11]  Lihui Wang,et al.  Scheduling in cloud manufacturing: state-of-the-art and research challenges , 2019, Int. J. Prod. Res..

[12]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[13]  Paulo E. Miyagi,et al.  Service Composition in the Cloud-Based Manufacturing Focused on the Industry 4.0 , 2015, DoCEIS.

[14]  Shane Legg,et al.  Deep Reinforcement Learning from Human Preferences , 2017, NIPS.

[15]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[16]  Liang Xu,et al.  Research on hybrid cloud particle swarm optimization for multi-objective flexible job shop scheduling problem , 2017, 2017 6th International Conference on Computer Science and Network Technology (ICCSNT).

[17]  George Q. Huang,et al.  A cooperative approach to service booking and scheduling in cloud manufacturing , 2019, Eur. J. Oper. Res..

[18]  Yongkui Liu,et al.  Enterprises in Cloud Manufacturing: A Preliminary Exploration , 2017 .

[19]  Benoît Iung,et al.  Challenges for the cyber-physical manufacturing enterprises of the future , 2019, Annu. Rev. Control..

[20]  Yi Lei,et al.  A niching behaviour-based algorithm for multi-level manufacturing service composition optimal-selection , 2020, J. Ambient Intell. Humaniz. Comput..

[21]  Anna Bogdan,et al.  The Resource Efficiency Assessment Technique for the Foundry Production , 2014 .

[22]  Lihui Wang,et al.  Cloud manufacturing: latest advancements and future trends , 2018 .

[23]  Feng Xiang,et al.  The case-library method for service composition and optimal selection of big manufacturing data in cloud manufacturing system , 2016 .

[24]  Nooruldeen Nasih Qader,et al.  Service load balancing, scheduling, and logistics optimization in cloud manufacturing by using genetic algorithm , 2019, Concurr. Comput. Pract. Exp..

[25]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[26]  Xifan Yao,et al.  Cloud Manufacturing Service Composition Optimization with Improved Genetic Algorithm , 2019, Mathematical Problems in Engineering.

[27]  Alexandre Dolgui,et al.  A dynamic model and an algorithm for short-term supply chain scheduling in the smart factory industry 4.0 , 2016 .

[28]  Xun Xu,et al.  Cloud manufacturing: key issues and future perspectives , 2019, Int. J. Comput. Integr. Manuf..