ProgCEP: A Programming Model for Complex Event Processing over Fog Infrastructure

Complex Event Processing (CEP) is a powerful paradigm that can derive meaningful insights by correlating multiple data sources, e.g., in the Internet-of-Things applications. However, these applications often require deployment across a wide variety of devices ranging from mobile devices to edge and cloud or simply put: fog infrastructure. This is not easily possible using existing programming models because of missing (i) support for deployment on heterogeneous devices and (ii) important interfaces for the deployment of CEP, e.g., for developing operator placement algorithms.AB@In this paper, we present ProgCEP: a programming model that facilitates the development of the operator placement algorithm and its deployment in a fog computing setting. In addition, it is portable and deployable on any kind of fog infrastructure and provides dynamic scaling of resources and deployment using the operator placement algorithm. We evaluate ProgCEP on its applicability and realizability on a publicly available fog testbed involving on-site, GENI and CloudLab resources using Docker tools. To this end, we enable (i) deployment of CEP using our dockerized implementation on the aforementioned fog infrastructure in less than 50 secs (on 25 distributed resources) and (ii) easy development of operator placement algorithms in terms of minimum lines of code.

[1]  Rajkumar Buyya,et al.  iFogSim: A toolkit for modeling and simulation of resource management techniques in the Internet of Things, Edge and Fog computing environments , 2016, Softw. Pract. Exp..

[2]  Vladimir Vlassov,et al.  SpanEdge: Towards Unifying Stream Processing over Central and Near-the-Edge Data Centers , 2016, 2016 IEEE/ACM Symposium on Edge Computing (SEC).

[3]  Frank Dürr,et al.  Solving the Multi-Operator Placement Problem in Large-Scale Operator Networks , 2010, 2010 Proceedings of 19th International Conference on Computer Communications and Networks.

[4]  Boris Koldehofe,et al.  Quality-Aware Runtime Adaptation in Complex Event Processing , 2017, 2017 IEEE/ACM 12th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS).

[5]  David Lillethun,et al.  Mobile fog: a programming model for large-scale applications on the internet of things , 2013, MCC '13.

[6]  Seif Haridi,et al.  Apache Flink™: Stream and Batch Processing in a Single Engine , 2015, IEEE Data Eng. Bull..

[7]  Ralf Steinmetz,et al.  Transitions for Increased Flexibility in Fog Computing: A Case Study on Complex Event Processing , 2019, Informatik Spektrum.

[8]  Ruben Mayer,et al.  EmuFog: Extensible and scalable emulation of large-scale fog computing infrastructures , 2017, 2017 IEEE Fog World Congress (FWC).

[9]  Jignesh M. Patel,et al.  Twitter Heron: Stream Processing at Scale , 2015, SIGMOD Conference.

[10]  M. Frans Kaashoek,et al.  Vivaldi: a decentralized network coordinate system , 2004, SIGCOMM 2004.

[11]  Thomas Plagemann,et al.  Operator placement for efficient distributed complex event processing in MANETs , 2015, 2015 IEEE 11th International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob).

[12]  Margo I. Seltzer,et al.  Network-Aware Operator Placement for Stream-Processing Systems , 2006, 22nd International Conference on Data Engineering (ICDE'06).

[13]  Boris Koldehofe,et al.  TCEP: Adapting to Dynamic User Environments by Enabling Transitions between Operator Placement Mechanisms , 2018, DEBS.

[14]  Vincenzo Grassi,et al.  Optimal operator placement for distributed stream processing applications , 2016, DEBS.

[15]  Hamid Reza Arkian,et al.  MIST: Fog-based data analytics scheme with cost-efficient resource provisioning for IoT crowdsensing applications , 2017, J. Netw. Comput. Appl..

[16]  Ying Li,et al.  Placement Strategies for Internet-Scale Data Stream Systems , 2008, IEEE Internet Computing.