Augmented Reality-Based Process Modelling for the Internet of Things with HoloFlows

Workflow technologies can be handy to model, execute and analyse simple processes in Internet of Things (IoT) environments. End-users are enabled to compose processes and thereby automate basic repetitive tasks involving one or more IoT devices. However, the modelling of these IoT workflows currently relies on rather bloated and complex desktop applications, deep knowledge of the underlying process notations and a high level of abstraction, which makes workflow modelling too complicated for end-users. In this work we propose to use augmented reality (AR) to simplify the modelling and configuration of IoT workflows. With our HoloFlows app for smart glasses end-users are able to explore their surrounding IoT environment and model various types of basic processes involving sensors and actuators by simply connecting two or more physical IoT devices via virtual wires. AR technology hereby facilitates the understanding of the physical contexts and relations among the IoT devices and provides a new and more intuitive way of modelling workflows in the cyber-physical world. We demonstrate the HoloFlows app with the help of various IoT workflows from the smart home domain.

[1]  Carsten Magerkurth,et al.  IoT Reference Model , 2013 .

[2]  Mathias Weske,et al.  Toward uBPMN-Based Patterns for Modeling Ubiquitous Business Processes , 2018, IEEE Transactions on Industrial Informatics.

[3]  P. Giner,et al.  Developing Mobile Business Processes for the Internet of Things , 2010, IEEE Pervasive Computing.

[4]  Dimitris Karagiannis,et al.  Service-Driven Enrichment for KbR in the OMiLAB Environment , 2017, ICServ.

[5]  Vasileios Theodorou,et al.  Goal-Based Semantic Queries for Dynamic Processes in the Internet of Things , 2016, Int. J. Semantic Comput..

[6]  Mike Bailey,et al.  Augment Your Reality , 2016, IEEE Computer Graphics and Applications.

[7]  Ricardo Martinho,et al.  Internet of Things Aware WS-BPEL Business Processes Context Variables and Expected Exceptions , 2014, J. Univers. Comput. Sci..

[8]  Florian Niebling,et al.  Modelling complex and flexible processes for smart cyber-physical environments , 2013, J. Comput. Sci..

[9]  Mark Weiser.  The computer for the 21st century , 1991 .

[10]  Thomas Schlegel,et al.  Toward an execution system for self-healing workflows in cyber-physical systems , 2018, Software & Systems Modeling.

[11]  Jan Recker,et al.  A Prototype Augmented Reality Collaborative Process Modelling Tool , 2011, BPM.

[12]  Marimuthu Palaniswami,et al.  Internet of Things (IoT): A vision, architectural elements, and future directions , 2012, Future Gener. Comput. Syst..

[13]  Francisco Martins,et al.  Modelling IoT behaviour within BPMN Business Processes , 2017 .

[14]  Slim Kallel,et al.  BPMN4CPS: A BPMN Extension for Modeling Cyber-Physical Systems , 2016, 2016 IEEE 25th International Conference on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE).

[15]  Jan Mendling,et al.  Challenges of smart business process management: An introduction to the special issue , 2017, Decis. Support Syst..

[16]  Ronald Azuma,et al.  Recent Advances in Augmented Reality , 2001, IEEE Computer Graphics and Applications.

[17]  Marcin Hewelt,et al.  Integrating IoT Devices into Business Processes , 2018, CAiSE Workshops.

[18]  Andrew Y. C. Nee,et al.  A comprehensive survey of augmented reality assembly research , 2016, Advances in Manufacturing.

[19]  Uwe Aßmann,et al.  HoloFacility: Get in Touch with Machines at Trade Fairs using Holograms , 2017, ISS.

[20]  Oliver Kopp,et al.  Extending BPMN for Wireless Sensor Networks , 2013, 2013 IEEE 15th Conference on Business Informatics.

[21]  Uwe Aßmann,et al.  Mixed reality cyber-physical systems control and workflow composition , 2017, MUM.

[22]  Manfred Reichert,et al.  BPM to Go: Supporting Business Processes in a Mobile and Sensing World , 2015 .

[23]  Sonja Meyer,et al.  Internet of Things-Aware Process Modeling: Integrating IoT Devices as Business Process Resources , 2013, CAiSE.

[24]  Rajkumar Buyya,et al.  Mobile Cloud Business Process Management System for the Internet of Things , 2015, ACM Comput. Surv..

[25]  Sébastien Gérard,et al.  Semantic Framework for Internet of Things-Aware Business Process Development , 2017, 2017 IEEE 26th International Conference on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE).

[26]  Rik Van de Walle,et al.  User-friendly configuration of smart environments , 2014, 2014 IEEE International Conference on Pervasive Computing and Communication Workshops (PERCOM WORKSHOPS).

[27]  Manfred Reichert,et al.  Towards Context-Aware Process Guidance in Cyber-Physical Systems with Augmented Reality , 2018, 2018 4th International Workshop on Requirements Engineering for Self-Adaptive, Collaborative, and Cyber Physical Systems (RESACS).