Revenue Models for Streaming Applications over Shared Clouds

When multiple users execute their streaming applications over a shared Cloud infrastructure, the provider typically captures the Quality of Service (QoS) for each application at a Service Level Agreement (SLA). Such an SLA identifies the cost that a user must pay to achieve the required QoS, and a penalty that must be paid to the user in case the QoS cannot be met. Assuming the maximisation of the revenue as the provider's objective, then it must decide: (i) which user streams to accept for storage and analysis; (ii) how many (computational / storage) resources to allocate to each stream in order to improve overall revenue and minimise cost. In this paper, we analyse revenue models for in-transit streaming applications, executed over a shared Cloud infrastructure under the presence of faulty computational resources. We propose an architecture that features a token bucket process envelop to accept user streams; and a control loop to enable resource allocation, while minimising operational cost.

[1]  Mark Greenwood,et al.  Taverna: lessons in creating a workflow environment for the life sciences: Research Articles , 2006 .

[2]  Ying Xing,et al.  Scalable Distributed Stream Processing , 2003, CIDR.

[3]  Edward A. Lee,et al.  CONCURRENCY AND COMPUTATION: PRACTICE AND EXPERIENCE Concurrency Computat.: Pract. Exper. 2000; 00:1–7 Prepared using cpeauth.cls [Version: 2002/09/19 v2.02] Taverna: Lessons in creating , 2022 .

[4]  R. Stephenson A and V , 1962, The British journal of ophthalmology.

[5]  Sharma Chakravarthy,et al.  Stream Data Processing: A Quality of Service Perspective - Modeling, Scheduling, Load Shedding, and Complex Event Processing , 2009, Advances in Database Systems.

[6]  D.E. Tolmie Gigabit networking , 1992, IEEE LTS.

[7]  Omer F. Rana,et al.  An uncoordinated asynchronous checkpointing model for hierarchical scientific workflows , 2010, J. Comput. Syst. Sci..

[8]  Omer F. Rana,et al.  Adaptive exception handling for scientific workflows , 2010 .

[9]  Omer F. Rana,et al.  Enforcing QoS in scientific workflow systems enacted over Cloud infrastructures , 2012, J. Comput. Syst. Sci..

[10]  Ian J. Taylor,et al.  The Triana Workflow Environment: Architecture and Applications , 2007, Workflows for e-Science, Scientific Workflows for Grids.

[11]  Sang-Min Park,et al.  Data throttling for data-intensive workflows , 2008, 2008 IEEE International Symposium on Parallel and Distributed Processing.

[12]  Ying Xing,et al.  The Design of the Borealis Stream Processing Engine , 2005, CIDR.

[13]  Domenico Talia,et al.  Characterizing quality of resilience in scientific workflows , 2011, WORKS '11.

[14]  Biörn Johan Biörnstad,et al.  A Workflow Approach to Stream Processing , 2008 .

[15]  Yogesh L. Simmhan,et al.  Adaptive rate stream processing for smart grid applications on clouds , 2011, ScienceCloud '11.

[16]  Patrick Valduriez,et al.  StreamCloud: A Large Scale Data Streaming System , 2010, 2010 IEEE 30th International Conference on Distributed Computing Systems.

[17]  Shawn Bowers,et al.  An approach for pipelining nested collections in scientific workflows , 2005, SGMD.

[18]  Omer F. Rana,et al.  Adaptive exception handling for scientific workflows , 2010, Concurr. Comput. Pract. Exp..

[19]  Y. Simmhan,et al.  Towards Reliable, Performant Workflows for Streaming-Applications on Cloud Platforms , 2011, 2011 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing.

[20]  Omer F. Rana,et al.  End-to-End QoS on Shared Clouds for Highly Dynamic, Large-Scale Sensing Data Streams , 2012, 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012).

[21]  Omer F. Rana,et al.  Autonomic streaming pipeline for scientific workflows , 2011, Concurr. Comput. Pract. Exp..

[22]  Opher Etzion,et al.  Event Processing in Action , 2010 .

[23]  G. Alonso,et al.  Parallel computing patterns for Grid workflows , 2006, 2006 Workshop on Workflows in Support of Large-Scale Science.