IoTSim-Stream: Modelling stream graph application in cloud simulation

In the era of big data, the high velocity of data imposes the demand for processing such data in real-time to gain real-time insights. Various real-time big data platforms/services (i.e. Apache Storm, Amazon Kinesis) allow to develop real-time big data applications to process continuous data to get incremental results. Composing those applications to form a workflow that is designed to accomplish certain goal is the becoming more important nowadays. However, given the current need of composing those applications into data pipelines forming stream workflow applications (aka stream graph applications) to support decision making, a simulation toolkit is required to simulate the behaviour of this graph application in Cloud computing environment. Therefore, in this paper, we propose an IoT Simulator for Stream processing on the big data (named IoTSim-Stream) that offers an environment to model complex stream graph applications in Multicloud environment, where the large-scale simulation-based studies can be conducted to evaluate and analyse these applications. The experimental results show that IoTSim-Stream is effective in modelling and simulating different structures of complex stream graph applications with excellent performance and scalability.

[1]  P. Mell,et al.  The NIST Definition of Cloud Computing , 2011 .

[2]  Roy H. Campbell,et al.  Play It Again, SimMR! , 2011, 2011 IEEE International Conference on Cluster Computing.

[3]  Hwangnam Kim,et al.  MR-CloudSim: Designing and implementing MapReduce computing model on CloudSim , 2012, 2012 International Conference on ICT Convergence (ICTC).

[4]  Maozhen Li,et al.  MRSim: A discrete event based MapReduce simulator , 2010, 2010 Seventh International Conference on Fuzzy Systems and Knowledge Discovery.

[5]  Rajkumar Buyya,et al.  NetworkCloudSim: Modelling Parallel Applications in Cloud Simulations , 2011, 2011 Fourth IEEE International Conference on Utility and Cloud Computing.

[6]  Kun-Lung Wu,et al.  IBM Streams Processing Language: Analyzing Big Data in motion , 2013, IBM J. Res. Dev..

[7]  Ellis Solaiman,et al.  Orchestrating BigData Analysis Workflows , 2017, IEEE Cloud Computing.

[8]  Guanying Wang,et al.  Using realistic simulation for performance analysis of mapreduce setups , 2009, LSAP '09.

[9]  Anita Weismantel Mikasa,et al.  Play it again , 1995 .

[10]  Mei-Hui Su,et al.  Characterization of scientific workflows , 2008, 2008 Third Workshop on Workflows in Support of Large-Scale Science.

[11]  N CalheirosRodrigo,et al.  CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms , 2011 .

[12]  Luiz Fernando Bittencourt,et al.  CEPSim: Modelling and simulation of Complex Event Processing systems in cloud environments , 2016, Future Gener. Comput. Syst..

[13]  Rajkumar Buyya,et al.  Article in Press Future Generation Computer Systems ( ) – Future Generation Computer Systems Cloud Computing and Emerging It Platforms: Vision, Hype, and Reality for Delivering Computing as the 5th Utility , 2022 .

[14]  Yonggang Wen,et al.  Toward Scalable Systems for Big Data Analytics: A Technology Tutorial , 2014, IEEE Access.

[15]  Prem Prakash Jayaraman,et al.  IOTSim: a Cloud based Simulator for Analysing IoT Applications , 2016, ArXiv.

[16]  Ewa Deelman,et al.  WorkflowSim: A toolkit for simulating scientific workflows in distributed environments , 2012, 2012 IEEE 8th International Conference on E-Science.