An IoT-based task scheduling optimization scheme considering the deadline and cost-aware scientific workflow for cloud computing

Large-scale applications of Internet of things (IoT), which require considerable computing tasks and storage resources, are increasingly deployed in cloud environments. Compared with the traditional computing model, characteristics of the cloud such as pay-as-you-go, unlimited expansion, and dynamic acquisition represent different conveniences for these applications using the IoT architecture. One of the major challenges is to satisfy the quality of service requirements while assigning resources to tasks. In this paper, we propose a deadline and cost-aware scheduling algorithm that minimizes the execution cost of a workflow under deadline constraints in the infrastructure as a service (IaaS) model. Considering the virtual machine (VM) performance variation and acquisition delay, we first divide tasks into different levels according to the topological structure so that no dependency exists between tasks at the same level. Three strings are used to code the genes in the proposed algorithm to better reflect the heterogeneous and resilient characteristics of cloud environments. Then, HEFT is used to generate individuals with the minimum completion time and cost. Novel schemes are developed for crossover and mutation to increase the diversity of the solutions. Based on this process, a task scheduling method that considers cost and deadlines is proposed. Experiments on workflows that simulate the structured tasks of the IoT demonstrate that our algorithm achieves a high success rate and performs well compared to state-of-the-art algorithms.

[1]  Jianli Pan,et al.  Future Edge Cloud and Edge Computing for Internet of Things Applications , 2018, IEEE Internet of Things Journal.

[2]  Rajkumar Buyya,et al.  Meeting Deadlines of Scientific Workflows in Public Clouds with Tasks Replication , 2014, IEEE Transactions on Parallel and Distributed Systems.

[3]  Marios D. Dikaiakos,et al.  Scheduling Workflows with Budget Constraints , 2007, Grid 2007.

[4]  Keqiu Li,et al.  How Can Heterogeneous Internet of Things Build Our Future: A Survey , 2018, IEEE Communications Surveys & Tutorials.

[5]  Dick H. J. Epema,et al.  Deadline-constrained workflow scheduling algorithms for Infrastructure as a Service Clouds , 2013, Future Gener. Comput. Syst..

[6]  Rajkumar Buyya,et al.  Multi-objective planning for workflow execution on Grids , 2007, 2007 8th IEEE/ACM International Conference on Grid Computing.

[7]  Bryan Ng,et al.  Budget and Deadline Aware e-Science Workflow Scheduling in Clouds , 2019, IEEE Transactions on Parallel and Distributed Systems.

[8]  Deo Prakash Vidyarthi,et al.  A Cost-Effective Deadline-Constrained Dynamic Scheduling Algorithm for Scientific Workflows in a Cloud Environment , 2018, IEEE Transactions on Cloud Computing.

[9]  Hilde van der Togt,et al.  Publisher's Note , 2003, J. Netw. Comput. Appl..

[10]  Jin-Soo Kim,et al.  BTS: Resource capacity estimate for time-targeted science workflows , 2011, J. Parallel Distributed Comput..

[11]  Jun Zhang,et al.  An Ant Colony Optimization Approach to a Grid Workflow Scheduling Problem With Various QoS Requirements , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[12]  Jorge-Arnulfo Quiané-Ruiz,et al.  Runtime measurements in the cloud , 2010, Proc. VLDB Endow..

[13]  Radu Prodan,et al.  Performance and cost optimization for multiple large-scale grid workflow applications , 2007, Proceedings of the 2007 ACM/IEEE Conference on Supercomputing (SC '07).

[14]  Jin-Soo Kim,et al.  Cost optimized provisioning of elastic resources for application workflows , 2011, Future Gener. Comput. Syst..

[15]  Eric Gossett,et al.  Big Data: A Revolution That Will Transform How We Live, Work, and Think , 2015 .

[16]  P. Mell,et al.  The NIST Definition of Cloud Computing , 2011 .

[17]  Yun Yang,et al.  Robust Scheduling of Scientific Workflows with Deadline and Budget Constraints in Clouds , 2014, 2014 IEEE 28th International Conference on Advanced Information Networking and Applications.

[18]  Xuyun Zhang,et al.  A QoS-aware virtual machine scheduling method for energy conservation in cloud-based cyber-physical systems , 2019, World Wide Web.

[19]  Jun Zhang,et al.  Deadline constrained cloud computing resources scheduling for cost optimization based on dynamic objective genetic algorithm , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[20]  Rajkumar Buyya,et al.  A Particle Swarm Optimization-Based Heuristic for Scheduling Workflow Applications in Cloud Computing Environments , 2010, 2010 24th IEEE International Conference on Advanced Information Networking and Applications.

[21]  Xiaohui Liu,et al.  Evolutionary Multi-Objective Workflow Scheduling in Cloud , 2016, IEEE Transactions on Parallel and Distributed Systems.

[22]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[23]  Jarek Nabrzyski,et al.  Cost- and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds , 2012, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis.

[24]  Rajkumar Buyya,et al.  Cost-based scheduling of scientific workflow applications on utility grids , 2005, First International Conference on e-Science and Grid Computing (e-Science'05).

[25]  G. Bruce Berriman,et al.  On the Use of Cloud Computing for Scientific Workflows , 2008, 2008 IEEE Fourth International Conference on eScience.

[26]  Ming Mao,et al.  A Performance Study on the VM Startup Time in the Cloud , 2012, 2012 IEEE Fifth International Conference on Cloud Computing.

[27]  Viktor Mayer-Schnberger,et al.  Big Data: A Revolution That Will Transform How We Live, Work, and Think , 2013 .

[28]  Robert E. Tarjan,et al.  Depth-First Search and Linear Graph Algorithms , 1972, SIAM J. Comput..

[29]  Rajkumar Buyya,et al.  Deadline Based Resource Provisioningand Scheduling Algorithm for Scientific Workflows on Clouds , 2014, IEEE Transactions on Cloud Computing.

[30]  Ann L. Chervenak,et al.  Characterizing and profiling scientific workflows , 2013, Future Gener. Comput. Syst..

[31]  Qiong Wu,et al.  The research of multimedia cloud computing platform data dynamic task scheduling optimization method in multi core environment , 2017, Multimedia Tools and Applications.

[32]  Marty Humphrey,et al.  Auto-scaling to minimize cost and meet application deadlines in cloud workflows , 2011, 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[33]  Manu Vardhan,et al.  Cost Effective Genetic Algorithm for Workflow Scheduling in Cloud Under Deadline Constraint , 2016, IEEE Access.

[34]  Tiago Ferra de Sousa,et al.  Particle Swarm based Data Mining Algorithms for classification tasks , 2004, Parallel Comput..

[35]  Rizos Sakellariou,et al.  Representing Variant Calling Format as Directed Acyclic Graphs to Enable the Use of Cloud Computing for Efficient and Cost Effective Genome Analysis , 2017, 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID).

[36]  Qingsheng Zhu,et al.  Fluctuation-Aware and Predictive Workflow Scheduling in Cost-Effective Infrastructure-as-a-Service Clouds , 2018, IEEE Access.

[37]  Randy H. Katz,et al.  A view of cloud computing , 2010, CACM.

[38]  Yongsheng Ding,et al.  Endocrine-based coevolutionary multi-swarm for multi-objective workflow scheduling in a cloud system , 2017, Soft Comput..

[39]  Qingbo Wu,et al.  Workflow scheduling in cloud: a survey , 2015, The Journal of Supercomputing.

[40]  Darrell Whitley,et al.  A genetic algorithm tutorial , 1994, Statistics and Computing.

[41]  Qiming Zou,et al.  Research on Cost-Driven Services Composition in an Uncertain Environment , 2019 .

[42]  Qingsheng Zhu,et al.  Deadline-Constrained Cost Optimization Approaches for Workflow Scheduling in Clouds , 2017, IEEE Transactions on Parallel and Distributed Systems.

[43]  Ian T. Foster,et al.  Cost-Aware Cloud Provisioning , 2015, 2015 IEEE 11th International Conference on e-Science.

[44]  Yucong Duan,et al.  Toward service selection for workflow reconfiguration: An interface-based computing solution , 2018, Future Gener. Comput. Syst..

[45]  Yuan Ying Bottom Level Based Heuristic for Workflow Scheduling in Grids , 2008 .

[46]  John Shalf,et al.  Performance Analysis of High Performance Computing Applications on the Amazon Web Services Cloud , 2010, 2010 IEEE Second International Conference on Cloud Computing Technology and Science.

[47]  Daniel Westreich,et al.  Big Data: A Revolution That Will Transform How We Live, Work, and Think , 2014 .