QL-HEFT: a novel machine learning scheduling scheme base on cloud computing environment

Cloud computing is a computing model that fully utilizes the resources on the Internet to maximize the utilization of resources. Due to a large number of users and tasks, it is important to achieve efficient scheduling of tasks submitted by users. Task scheduling is one of the crucial and challenging non-deterministic polynomial-hard problems in cloud computing. In task scheduling, obtaining shorter makespan is an important objective and is related to the pros and cons of the algorithm. Machine learning algorithms represent a new method for solving this type of problem. In this paper, we propose a novel task scheduling algorithm called QL-HEFT that combines Q -learning with the heterogeneous earliest finish time (HEFT) algorithm to reduce the makespan. The algorithm uses the upward rank ( rank u ) value of HEFT as the immediate reward in the Q -learning framework. The agent can obtain better learning results to update the Q -table through the self-learning process. The QL-HEFT algorithm is divided into two major phases: a task sorting phase based on Q -learning for obtaining an optimal order and a processor allocation phase using the earliest finish time strategy. Experiments show that QL-HEFT achieves a shorter makespan compared to three other classical scheduling algorithms as well as good performances in terms of the average response time.

[1]  Kenli Li,et al.  A Parallel Multiclassification Algorithm for Big Data Using an Extreme Learning Machine , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[2]  Salim Hariri,et al.  Performance-Effective and Low-Complexity Task Scheduling for Heterogeneous Computing , 2002, IEEE Trans. Parallel Distributed Syst..

[3]  Shahaboddin Shamshirband,et al.  Static Task Scheduling in Cooperative Distributed Systems Based on Soft Computing Techniques , 2010 .

[4]  Laurence T. Yang,et al.  List-Based Task Scheduling for Cloud Computing , 2016, 2016 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData).

[5]  Bryan Ng,et al.  Scheduling deadline constrained scientific workflows on dynamically provisioned cloud resources , 2017, Future Gener. Comput. Syst..

[6]  Zhiping Peng,et al.  Multiple DAGs Workflow Scheduling Algorithm Based on Reinforcement Learning in Cloud Computing , 2015, ISICA.

[7]  Kun Wang,et al.  An improved binary PSO-based task scheduling algorithm in green cloud computing , 2014, 9th International Conference on Communications and Networking in China.

[8]  Kenli Li,et al.  An intermediate data placement algorithm for load balancing in Spark computing environment , 2018, Future Gener. Comput. Syst..

[9]  Kenli Li,et al.  A Fund-Constrained Investment Scheme for Profit Maximization in Cloud Computing , 2018, IEEE Transactions on Services Computing.

[10]  Kenli Li,et al.  Self‐adaptation and mutual adaptation for distributed scheduling in benevolent clouds , 2017, Concurr. Comput. Pract. Exp..

[11]  Kenli Li,et al.  Proactive scheduling in distributed computing - A reinforcement learning approach , 2014, J. Parallel Distributed Comput..

[12]  S. Haykin,et al.  A Q-learning-based dynamic channel assignment technique for mobile communication systems , 1999 .

[13]  Eduardo F. Morales,et al.  An Introduction to Reinforcement Learning , 2011 .

[14]  Zhenchun Wei,et al.  A task scheduling algorithm based on Q-learning and shared value function for WSNs , 2017, Comput. Networks.

[15]  Florin Pop,et al.  New scheduling approach using reinforcement learning for heterogeneous distributed systems , 2017, J. Parallel Distributed Comput..

[16]  D. Sirisha,et al.  Minimal Start Time Heuristics for Scheduling Workflows in Heterogeneous Computing Systems , 2016, ICDCIT.

[17]  Kenli Li,et al.  A genetic algorithm for task scheduling on heterogeneous computing systems using multiple priority queues , 2014, Inf. Sci..

[18]  Junwei Cao,et al.  A Case Study on the Use of Workflow Technologies for Scientific Analysis: Gravitational Wave Data Analysis , 2007, Workflows for e-Science, Scientific Workflows for Grids.

[19]  Kenli Li,et al.  A DAG Task Scheduling Scheme on Heterogeneous Computing Systems Using Invasive Weed Optimization Algorithm , 2014, 2014 Sixth International Symposium on Parallel Architectures, Algorithms and Programming.

[20]  Mehdi Kargahi,et al.  Ant colony based constrained workflow scheduling for heterogeneous computing systems , 2016, Cluster Computing.

[21]  Bo Xu,et al.  A Reinforcement Learning-Based Mixed Job Scheduler Scheme for Grid or IaaS Cloud , 2020, IEEE Transactions on Cloud Computing.

[22]  Cheng-Zhong Xu,et al.  URL: A unified reinforcement learning approach for autonomic cloud management , 2012, J. Parallel Distributed Comput..

[23]  Zheng Xiao,et al.  earning non-cooperative game for load balancing under elf-interested distributed environment , 2017 .

[24]  Geoffrey Zweig,et al.  The microsoft 2016 conversational speech recognition system , 2016, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[25]  Kenli Li,et al.  A novel task scheduling scheme in a cloud computing environment using hybrid biogeography-based optimization , 2019, Soft Comput..

[26]  Jan Peters,et al.  Reinforcement learning in robotics: A survey , 2013, Int. J. Robotics Res..

[27]  Weiqin Tong,et al.  Task scheduling of cloud computing based on Improved CHC algorithm , 2014, 2014 International Conference on Audio, Language and Image Processing.

[28]  Keqin Li,et al.  Energy and time constrained task scheduling on multiprocessor computers with discrete speed levels , 2016, J. Parallel Distributed Comput..

[29]  Li Zhao,et al.  Managing Large-Scale Workflow Execution from Resource Provisioning to Provenance Tracking: The CyberShake Example , 2006, 2006 Second IEEE International Conference on e-Science and Grid Computing (e-Science'06).

[30]  Mohit Agarwal,et al.  A genetic algorithm inspired task scheduling in cloud computing , 2016, 2016 International Conference on Computing, Communication and Automation (ICCCA).

[31]  Sendhil Mullainathan,et al.  Machine Learning: An Applied Econometric Approach , 2017, Journal of Economic Perspectives.

[32]  Ewa Deelman,et al.  WorkflowSim: A toolkit for simulating scientific workflows in distributed environments , 2012, 2012 IEEE 8th International Conference on E-Science.

[33]  Randy H. Katz,et al.  A view of cloud computing , 2010, CACM.

[34]  Hamid Arabnejad,et al.  List Scheduling Algorithm for Heterogeneous Systems by an Optimistic Cost Table , 2014, IEEE Transactions on Parallel and Distributed Systems.

[35]  Rajkumar Buyya,et al.  CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms , 2011, Softw. Pract. Exp..

[36]  Tahani Aladwani.  Improving Tasks Scheduling Performance in Cloud Computing Environment by Using Analytic Hierarchy Process Model , 2017, 2017 International Conference on Green Informatics (ICGI).

[37]  Kenli Li,et al.  A New Cloud Service Mechanism for Profit Optimizations of a Cloud Provider and Its Users , 2017 .

[38]  Weiwei Lin,et al.  Random task scheduling scheme based on reinforcement learning in cloud computing , 2015, Cluster Computing.

[39]  Xizhao Wang,et al.  An On-line Multi-CBR Agent Dispatching Algorithm , 2006, Soft Comput..

[40]  Kenli Li,et al.  Minimizing Cost of Scheduling Tasks on Heterogeneous Multicore Embedded Systems , 2016, ACM Trans. Embed. Comput. Syst..

[41]  Ann L. Chervenak,et al.  Characterizing and profiling scientific workflows , 2013, Future Gener. Comput. Syst..

[42]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[43]  Mei-Hui Su,et al.  Characterization of scientific workflows , 2008, 2008 Third Workshop on Workflows in Support of Large-Scale Science.

[44]  Olivier Barais,et al.  Investigating Machine Learning Algorithms for Modeling SSD I/O Performance for Container-Based Virtualization , 2019, IEEE Transactions on Cloud Computing.

[45]  Navpreet Kaur Walia,et al.  A Review: Cloud Computing using Various Task Scheduling Algorithms , 2016 .

[46]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[47]  Chen Hongsong,et al.  Analyzing trust concerns in public clouds using finite state automata , 2016, 2016 2nd International Conference on Cloud Computing and Internet of Things (CCIOT).

[48]  Yoshua Bengio,et al.  Attention-Based Models for Speech Recognition , 2015, NIPS.

[49]  Hui Liu,et al.  HSIP: A Novel Task Scheduling Algorithm for Heterogeneous Computing , 2016, Sci. Program..

[50]  Milad Moradi,et al.  A centralized reinforcement learning method for multi-agent job scheduling in Grid , 2016, 2016 6th International Conference on Computer and Knowledge Engineering (ICCKE).

[51]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[52]  Avita Katal,et al.  An Optimized Task Scheduling Algorithm in Cloud Computing , 2016, 2016 IEEE 6th International Conference on Advanced Computing (IACC).

[53]  Aurko Roy,et al.  Reinforcement Learning under Model Mismatch , 2017, NIPS.