Managing geo-distributed stream processing pipelines for the IIoT with StreamPipes edge extensions

The industrial IoT and its promise to realize data-driven decision-making by analyzing industrial event streams is an important innovation driver in the industrial sector. Due to an enormous increase of generated data and the development of specialized hardware, new decentralized paradigms such as fog computing arised to overcome shortcomings of centralized cloud-only approaches. However, current undertakings are focused on static deployments of standalone services, which is insufficient for geo-distributed applications that are composed of multiple event-driven functions. In this paper, we present StreamPipes Edge Extensions (SEE), a novel contribution to the open source IIoT toolbox Apache StreamPipes. With SEE, domain experts are able to create stream processing pipelines in a graphical editor and to assign individual pipeline elements to available edge nodes, while underlying provisioning and deployment details are abstracted by the framework. The main contributions are (i) a fog cluster management model to represent computing node characteristics, (ii) a node controller for pipeline element life cycle management and (iii) a management framework to deploy event-driven functions to registered nodes. Our approach was validated in a real industrial setup showing low overall overhead of SEE as part of a robot-assisted product quality inspection use case.

[1]  Olivier Terzo,et al.  Heterogeneous Computing Architectures : Challenges and Vision , 2019 .

[2]  Manish Parashar,et al.  Data-Driven Stream Processing at the Edge , 2017, 2017 IEEE 1st International Conference on Fog and Edge Computing (ICFEC).

[3]  Jason P. Jue,et al.  All One Needs to Know about Fog Computing and Related Edge Computing Paradigms , 2019 .

[4]  Mohamed Mohamed,et al.  Foggy: A Framework for Continuous Automated IoT Application Deployment in Fog Computing , 2017, 2017 IEEE International Conference on AI & Mobile Services (AIMS).

[5]  Stefan Schulte,et al.  A Framework for Optimization, Service Placement, and Runtime Operation in the Fog , 2018, 2018 IEEE/ACM 11th International Conference on Utility and Cloud Computing (UCC).

[6]  Soumya Kanti Datta,et al.  Comparison of edge computing implementations: Fog computing, cloudlet and mobile edge computing , 2017, 2017 Global Internet of Things Summit (GIoTS).

[7]  Raja Lavanya,et al.  Fog Computing and Its Role in the Internet of Things , 2019, Advances in Computer and Electrical Engineering.

[8]  Boris Koldehofe,et al.  ProgCEP: A Programming Model for Complex Event Processing over Fog Infrastructure , 2019, DFSD '19.

[9]  Nenad Stojanovic,et al.  SEPP: Semantics-Based Management of Fast Data Streams , 2014, 2014 IEEE 7th International Conference on Service-Oriented Computing and Applications.

[10]  Frank Leymann,et al.  Deployment of Distributed Applications Across Public and Private Networks , 2019, 2019 IEEE 23rd International Enterprise Distributed Object Computing Conference (EDOC).

[11]  Kin K. Leung,et al.  Online Placement of Multi-Component Applications in Edge Computing Environments , 2016, IEEE Access.

[12]  Enrique Saurez,et al.  Incremental deployment and migration of geo-distributed situation awareness applications in the fog , 2016, DEBS.

[13]  Amir Karamoozian,et al.  On the Fog-Cloud Cooperation: How Fog Computing can address latency concerns of IoT applications , 2019, 2019 Fourth International Conference on Fog and Mobile Edge Computing (FMEC).

[14]  Patrick Wiener,et al.  StreamPipes Connect: Semantics-Based Edge Adapters for the IIoT , 2020, ESWC.

[15]  Frédéric Desprez,et al.  An Overview of Service Placement Problem in Fog and Edge Computing , 2020, ACM Comput. Surv..

[16]  Victor C. M. Leung,et al.  Developing IoT applications in the Fog: A Distributed Dataflow approach , 2015, 2015 5th International Conference on the Internet of Things (IOT).

[17]  Francesco De Pellegrini,et al.  Foggy: A Platform for Workload Orchestration in a Fog Computing Environment , 2017, 2017 IEEE International Conference on Cloud Computing Technology and Science (CloudCom).

[18]  Gürkan Solmaz,et al.  FogFlow: Easy Programming of IoT Services Over Cloud and Edges for Smart Cities , 2018, IEEE Internet of Things Journal.

[19]  Srinath Perera,et al.  Recent Advancements in Event Processing , 2018, ACM Comput. Surv..

[20]  Patrick Wiener,et al.  Towards Context-Aware and Dynamic Management of Stream Processing Pipelines for Fog Computing , 2019, 2019 IEEE 3rd International Conference on Fog and Edge Computing (ICFEC).

[21]  Daniel Grosu,et al.  Placement of Multi-Component Applications in Edge Computing Systems , 2017 .

[22]  Antonio Brogi,et al.  How to place your apps in the fog: State of the art and open challenges , 2019, Softw. Pract. Exp..

[23]  Ralph Johnson,et al.  design patterns elements of reusable object oriented software , 2019 .

[24]  Antonio Brogi,et al.  Container-Based Support for Autonomic Data Stream Processing Through the Fog , 2017, Euro-Par Workshops.