Temporal Verification for Scientific Cloud Workflows: State-of-the-Art and Research Challenges

Cloud computing is establishing itself as the latest computing paradigm in recent years. As doing science in the cloud is becoming a reality, scientists are now able to access public cloud centers and employ high-performance computing resources to run scientific applications. However, due to the dynamic nature of the cloud environment, the usability of scientific cloud workflow systems can be significantly deteriorated if without effective service quality assurance strategies. Specifically, workflow temporal verification as the major approach for workflow temporal QoS (Quality of Service) assurance plays a critical role in the on-time completion of large-scale scientific workflows. Great efforts have been dedicated to the area of workflow temporal verification in recent years and it is high time that we should define the key research issues for scientific cloud workflows in order to keep our research on the right track. In this paper, we systematically investigate this problem and present four key research issues based on the introduction of a generic temporal verification framework. Meanwhile, state-of-the-art solutions for each research issue and open challenges are also presented. Finally, SwinDeW-V, an ongoing research project on temporal verification as part of our SwinDeW-C cloud workflow system, is also demonstrated.

[1]  Jinjun Chen,et al.  Multiple states based temporal consistency for dynamic verification of fixed‐time constraints in Grid workflow systems , 2007, Concurr. Comput. Pract. Exp..

[2]  Radu Prodan,et al.  Overhead Analysis of Scientific Workflows in Grid Environments , 2008, IEEE Transactions on Parallel and Distributed Systems.

[3]  Xiao Liu,et al.  A Probabilistic Strategy for Setting Temporal Constraints in Scientific Workflows , 2008, BPM.

[4]  Xiao Liu,et al.  An Effective Framework of Light-Weight Handling for Three-Level Fine-Grained Recoverable Temporal Violations in Scientific Workflows , 2010, 2010 IEEE 16th International Conference on Parallel and Distributed Systems.

[5]  Xiao Liu,et al.  SwinDeW-C: A Peer-to-Peer Based Cloud Workflow System , 2010, Handbook of Cloud Computing.

[6]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[7]  Jinjun Chen,et al.  Activity Completion Duration Based Checkpoint Selection for Dynamic Verification of Temporal Constraints in Grid Workflow Systems , 2008, Int. J. High Perform. Comput. Appl..

[8]  Daniel Moldovan,et al.  MELA: Monitoring and Analyzing Elasticity of Cloud Services , 2013, 2013 IEEE 5th International Conference on Cloud Computing Technology and Science.

[9]  Priya Narasimhan,et al.  Service-Oriented Computing - ICSOC 2007, Fifth International Conference, Vienna, Austria, September 17-20, 2007, Proceedings , 2007, ICSOC.

[10]  Jinjun Chen,et al.  Temporal dependency-based checkpoint selection for dynamic verification of temporal constraints in scientific workflow systems , 2011, TSEM.

[11]  Yong Zhao,et al.  Cloud Computing and Grid Computing 360-Degree Compared , 2008, GCE 2008.

[12]  Xiao Liu,et al.  Selecting checkpoints along the time line: A novel temporal checkpoint selection strategy for monitoring a batch of parallel business processes , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[13]  Jun Zhang,et al.  An Ant Colony Optimization Approach to a Grid Workflow Scheduling Problem With Various QoS Requirements , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[14]  Yoichi Muraoka,et al.  Extended forecast of CPU and network load on computational Grid , 2004, IEEE International Symposium on Cluster Computing and the Grid, 2004. CCGrid 2004..

[15]  Wil M. P. van der Aalst,et al.  Workflow Exception Patterns , 2006, CAiSE.

[16]  Xiao Liu,et al.  A novel general framework for automatic and cost-effective handling of recoverable temporal violations in scientific workflow systems , 2011, J. Syst. Softw..

[17]  Rajkumar Buyya,et al.  SLA-Based Advance Reservations with Flexible and Adaptive Time QoS Parameters , 2007, ICSOC.

[18]  Jelena V. Misic,et al.  Performance Analysis of Cloud Computing Centers Using M/G/m/m+r Queuing Systems , 2012, IEEE Transactions on Parallel and Distributed Systems.

[19]  Xiao Liu,et al.  A Generic QoS Framework for Cloud Workflow Systems , 2011, 2011 IEEE Ninth International Conference on Dependable, Autonomic and Secure Computing.

[20]  Hector Garcia-Molina,et al.  Deadline Assignment in a Distributed Soft Real-Time System , 1997, IEEE Trans. Parallel Distributed Syst..

[21]  Xiao Liu,et al.  A probabilistic strategy for temporal constraint management in scientific workflow systems , 2011, Concurr. Comput. Pract. Exp..

[22]  Xiao Liu,et al.  A data placement strategy in scientific cloud workflows , 2010, Future Gener. Comput. Syst..

[23]  Xiao Liu,et al.  Handling Recoverable Temporal Violations in Scientific Workflow Systems: A Workflow Rescheduling Based Strategy , 2010, 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing.

[24]  Yuan-Chun Jiang,et al.  Preventing Temporal Violations in Scientific Workflows: Where and How , 2011, IEEE Transactions on Software Engineering.

[25]  Yun Yang,et al.  Temporal QOS Management in Scientific Cloud Workflow Systems , 2012 .

[26]  Rajkumar Buyya,et al.  A Taxonomy of Workflow Management Systems for Grid Computing , 2005, Proceedings of the 38th Annual Hawaii International Conference on System Sciences.

[27]  Yun Yang,et al.  A Scientific Cloud Workflow System , 2012 .

[28]  Jinjun Chen,et al.  Dynamic verification of temporal constraints on-the-fly for workflow systems , 2004, 11th Asia-Pacific Software Engineering Conference.

[29]  Xiao Liu,et al.  The Design of Cloud Workflow Systems , 2012, SpringerBriefs in Computer Science.

[30]  Jinjun Chen,et al.  Adaptive selection of necessary and sufficient checkpoints for dynamic verification of temporal constraints in grid workflow systems , 2007, TAAS.

[31]  Dennis Gannon,et al.  Workflows for e-Science, Scientific Workflows for Grids , 2014 .

[32]  Xiao Liu,et al.  Forecasting Duration Intervals of Scientific Workflow Activities Based on Time-Series Patterns , 2008, 2008 IEEE Fourth International Conference on eScience.

[33]  Jun Zhang,et al.  Workflow scheduling in grids: an ant colony optimization approach , 2007, 2007 IEEE Congress on Evolutionary Computation.

[34]  Jinjun Chen,et al.  Localising temporal constraints in scientific workflows , 2010, J. Comput. Syst. Sci..

[35]  Dennis Gannon,et al.  Scientific versus Business Workflows , 2007, Workflows for e-Science, Scientific Workflows for Grids.

[36]  Francine Berman,et al.  New Grid Scheduling and Rescheduling Methods in the GrADS Project , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[37]  Xiao Liu,et al.  Do we need to handle every temporal violation in scientific workflow systems? , 2014, TSEM.

[38]  Johann Eder,et al.  Time Constraints in Workflow Systems , 2013, Seminal Contributions to Information Systems Engineering.

[39]  Rajkumar Buyya,et al.  Market-Oriented Cloud Computing: Vision, Hype, and Reality for Delivering IT Services as Computing Utilities , 2008, 2008 10th IEEE International Conference on High Performance Computing and Communications.

[40]  Weisong Shi,et al.  An Adaptive Rescheduling Strategy for Grid Workflow Applications , 2007, 2007 IEEE International Parallel and Distributed Processing Symposium.

[41]  Ming Mao,et al.  A Performance Study on the VM Startup Time in the Cloud , 2012, 2012 IEEE Fifth International Conference on Cloud Computing.

[42]  Ian J. Taylor,et al.  Workflows and e-Science: An overview of workflow system features and capabilities , 2009, Future Gener. Comput. Syst..

[43]  Maria E. Orlowska,et al.  On Modeling and Verification of Temporal Constraints in Production Workflows , 1999, Knowledge and Information Systems.

[44]  G. Bruce Berriman,et al.  On the Use of Cloud Computing for Scientific Workflows , 2008, 2008 IEEE Fourth International Conference on eScience.

[45]  Francine Berman,et al.  New Grid Scheduling and Rescheduling Methods in the GrADS Project , 2004, IPDPS Next Generation Software Program - NSFNGS - PI Workshop.

[46]  Yuan-Chun Jiang,et al.  A novel statistical time-series pattern based interval forecasting strategy for activity durations in workflow systems , 2011, J. Syst. Softw..

[47]  J. Leon Zhao,et al.  A collaborative scheduling approach for service-driven scientific workflow execution , 2010, J. Comput. Syst. Sci..

[48]  José A. B. Fortes,et al.  CloudBLAST: Combining MapReduce and Virtualization on Distributed Resources for Bioinformatics Applications , 2008, 2008 IEEE Fourth International Conference on eScience.

[49]  Borko Furht,et al.  Handbook of Cloud Computing , 2010 .

[50]  Rajkumar Buyya,et al.  Workflow scheduling algorithms for grid computing , 2008 .