A Novel Reinforcement-Learning-Based Approach to Scientific Workflow Scheduling

Recently, the Cloud Computing paradigm is becoming increasingly popular in supporting large-scale and complex workflow applications. The workflow scheduling problem, which refers to finding the most suitable resource for each task of the workflow to meet user defined quality of service (QoS), attracts considerable research attention. Multi-objective optimization algorithms in workflow scheduling have many limitations, e.g., the encoding schemes in most existing heuristic-based scheduling algorithms require prior experts' knowledge and thus they can be ineffective when scheduling workflows upon dynamic cloud infrastructures with real-time. To address this problem, we propose a novel Reinforcement-Learning-Based algorithm to multi-workflow scheduling over IaaS clouds. The proposed algorithm aims at optimizing make-span and dwell time and is to achieve a unique set of correlated equilibrium solution. In the experiment, our algorithm is evaluated for famous scientific workflow templates and real-world industrial IaaS cloud platforms by a simulation process and we compare our algorithm to the current state-of-the-art heuristic algorithms, e.g., NSGA-II, MOPSO, GTBGA. The result shows that our algorithm performs better than compared algorithm.

[1]  Yusen Zhan,et al.  Theoretically-Grounded Policy Advice from Multiple Teachers in Reinforcement Learning Settings with Applications to Negative Transfer , 2016, IJCAI.

[2]  Jianping Fan,et al.  Leveraging Content Sensitiveness and User Trustworthiness to Recommend Fine-Grained Privacy Settings for Social Image Sharing , 2018, IEEE Transactions on Information Forensics and Security.

[3]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[4]  Weinan Zhang,et al.  MAgent: A Many-Agent Reinforcement Learning Platform for Artificial Collective Intelligence , 2017, AAAI.

[5]  Keith B. Hall,et al.  Correlated Q-Learning , 2003, ICML.

[6]  Kenli Li,et al.  Bi-objective workflow scheduling of the energy consumption and reliability in heterogeneous computing systems , 2017, Inf. Sci..

[7]  Jin Sun,et al.  Minimizing cost and makespan for workflow scheduling in cloud using fuzzy dominance sort based HEFT , 2019, Future Gener. Comput. Syst..

[8]  Alexander Boukhanovsky,et al.  Hybrid Evolutionary Workflow Scheduling Algorithm for Dynamic Heterogeneous Distributed Computational Environment , 2014, SOCO-CISIS-ICEUTE.

[9]  Y. Rui,et al.  Learning to Rank Using User Clicks and Visual Features for Image Retrieval , 2015, IEEE Transactions on Cybernetics.

[10]  Qingsheng Zhu,et al.  Fluctuation-Aware and Predictive Workflow Scheduling in Cost-Effective Infrastructure-as-a-Service Clouds , 2018, IEEE Access.

[11]  MengChu Zhou,et al.  Stochastic Modeling and Performance Analysis of Migration-Enabled and Error-Prone Clouds , 2015, IEEE Transactions on Industrial Informatics.

[12]  Prasanta K. Jana,et al.  A GSA based hybrid algorithm for bi-objective workflow scheduling in cloud computing , 2018, Future Gener. Comput. Syst..

[13]  Albert Y. Zomaya,et al.  Computation Offloading for Service Workflow in Mobile Cloud Computing , 2015, IEEE Transactions on Parallel and Distributed Systems.

[14]  Saeed Sharifian,et al.  A distributed load balancing and admission control algorithm based on Fuzzy type-2 and Game theory for large-scale SaaS cloud architectures , 2018, Future Gener. Comput. Syst..

[15]  Changyin Sun,et al.  An adaptive strategy via reinforcement learning for the prisoner U+02BC s dilemma game , 2018, IEEE/CAA Journal of Automatica Sinica.

[16]  MengChu Zhou,et al.  A Stochastic Approach to Analysis of Energy-Aware DVS-Enabled Cloud Datacenters , 2015, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[17]  Xiaofeng Liao,et al.  Reinforcement Learning for Constrained Energy Trading Games With Incomplete Information , 2017, IEEE Transactions on Cybernetics.

[18]  Shijun Liu,et al.  A Reinforcement Learning Based Workflow Application Scheduling Approach in Dynamic Cloud Environment , 2017, CollaborateCom.

[19]  Yunni Xia,et al.  Multi-Objective Optimization for Location Prediction of Mobile Devices in Sensor-Based Applications , 2018, IEEE Access.

[20]  Lei Wu,et al.  Scheduling Multi-Workflows Over Heterogeneous Virtual Machines With a Multi-Stage Dynamic Game-Theoretic Approach , 2018, Int. J. Web Serv. Res..

[21]  Yueshen Xu,et al.  Network Location-Aware Service Recommendation with Random Walk in Cyber-Physical Systems , 2017, Sensors.

[22]  Sanyam Kapoor,et al.  Multi-Agent Reinforcement Learning: A Report on Challenges and Approaches , 2018, ArXiv.

[23]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[24]  Carlos A. Coello Coello,et al.  Handling multiple objectives with particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[25]  Lenz Belzner,et al.  Optimization of global production scheduling with deep reinforcement learning , 2018 .

[26]  Albert Y. Zomaya,et al.  GA-ETI: An enhanced genetic algorithm for the scheduling of scientific workflows in cloud environments , 2016, J. Comput. Sci..

[27]  Xiao Liu,et al.  A sufficient and necessary temporal violation handling point selection strategy in cloud workflow , 2018, Future Gener. Comput. Syst..

[28]  Li Qirui,et al.  A Multi-object Optimization Cloud Workflow Scheduling Algorithm Based on Reinforcement Learning , 2018, ICIC 2018.

[29]  Dimitri P. Bertsekas,et al.  Feature-based aggregation and deep reinforcement learning: a survey and some new implementations , 2018, IEEE/CAA Journal of Automatica Sinica.

[30]  Xiaorong Li,et al.  Multi-Objective Game Theoretic Schedulingof Bag-of-Tasks Workflows on Hybrid Clouds , 2014, IEEE Transactions on Cloud Computing.

[31]  Srikanth Kandula,et al.  Resource Management with Deep Reinforcement Learning , 2016, HotNets.

[32]  Zhiping Peng,et al.  Multiple DAGs Workflow Scheduling Algorithm Based on Reinforcement Learning in Cloud Computing , 2015, ISICA.

[33]  Yueshen Xu,et al.  Collaborative Service Selection via Ensemble Learning in Mixed Mobile Network Environments , 2017, Entropy.

[34]  Sakshi Kaushal,et al.  A hybrid multi-objective Particle Swarm Optimization for scientific workflow scheduling , 2017, Parallel Comput..

[35]  Jianping Fan,et al.  iPrivacy: Image Privacy Protection by Identifying Sensitive Objects via Deep Multi-Task Learning , 2017, IEEE Transactions on Information Forensics and Security.

[36]  Sanjay Kadam,et al.  A novel multi-objective bacteria foraging optimization algorithm (MOBFOA) for multi-objective scheduling , 2018, Appl. Soft Comput..

[37]  Yi Wu,et al.  Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments , 2017, NIPS.