A Taxonomy for Combining Activity Recognition and Process Discovery in Industrial Environments

Despite the increasing automation levels in an Industry 4.0 scenario, the tacit knowledge of highly skilled manufacturing workers remains of strategic importance. Retaining this knowledge by formally capturing it is a challenge for industrial organisations. This paper explores research on automatically capturing this knowledge by using methods from activity recognition and process mining on data obtained from sensorised workers and environments. Activity recognition lifts the abstraction level of sensor data to recognizable activities and process mining methods discover models of process executions. We classify the existing work, which largely neglects the possibility of applying process mining, and derive a taxonomy that identifies challenges and research gaps.

[1]  Massimo Mecella,et al.  Automated Discovery of Process Models from Event Logs: Review and Benchmark , 2017, IEEE Transactions on Knowledge and Data Engineering.

[2]  Theodora A. Varvarigou,et al.  A top-down event-driven approach for concurrent activity recognition , 2012, Multimedia Tools and Applications.

[3]  Bernt Schiele,et al.  Remember and transfer what you have learned - recognizing composite activities based on activity spotting , 2010, International Symposium on Wearable Computers (ISWC) 2010.

[4]  Paul Lukowicz,et al.  Activity Recognition of Assembly Tasks Using Body-Worn Microphones and Accelerometers , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Peter Loos,et al.  Technology-Enhanced Process Elicitation of Worker Activities in Manufacturing , 2017, Business Process Management Workshops.

[6]  Mario Cannataro,et al.  Protein-to-protein interactions: Technologies, databases, and algorithms , 2010, CSUR.

[7]  Daniel Roggen,et al.  Event-Based Activity Tracking in Work Environments , 2006 .

[8]  Claudio Bettini,et al.  The Privacy Implications of Cyber Security Systems , 2018, ACM Comput. Surv..

[9]  Bodo Urban,et al.  Using Activity Recognition for the Tracking of Assembly Processes : Challenges and Requirements , 2014 .

[10]  Deborah Estrin,et al.  Improving activity classification for health applications on mobile devices using active and semi-supervised learning , 2010, 2010 4th International Conference on Pervasive Computing Technologies for Healthcare.

[11]  Philipp Scholl,et al.  Detecting Process Transitions from Wearable Sensors: An Unsupervised Labeling Approach , 2017, iWOAR.

[12]  Sebti Foufou,et al.  Intention recognition in manufacturing applications , 2015 .

[13]  J. Miura,et al.  Human-Robot Collaborative Assembly by On-line Human Action Recognition Based on an FSM Task Model , 2013 .

[14]  Sobah Abbas Petersen,et al.  Privacy Challenges for Process Mining in Human-Centered Industrial Environments , 2018, 2018 14th International Conference on Intelligent Environments (IE).

[15]  Paul Lukowicz,et al.  Recognizing Workshop Activity Using Body Worn Microphones and Accelerometers , 2004, Pervasive.

[16]  Franco Failli,et al.  An Integrated Environment Based on Augmented Reality and Sensing Device for Manual Assembly Workstations , 2016 .

[17]  Peter Loos,et al.  Activity Monitoring Using Wearable Sensors in Manual Production Processes-An Application of CPS for Automated Ergonomic Assessments , 2017 .

[18]  Hajo A. Reijers,et al.  Balanced multi-perspective checking of process conformance , 2016, Computing.

[19]  Sebastian Bader,et al.  Tracking Assembly Processes and Providing Assistance in Smart Factories , 2014, ICAART.

[20]  Xiaohui Peng,et al.  Deep Learning for Sensor-based Activity Recognition: A Survey , 2017, Pattern Recognit. Lett..

[21]  Alois Knoll,et al.  Human activity recognition in the context of industrial human-robot interaction , 2014, Signal and Information Processing Association Annual Summit and Conference (APSIPA), 2014 Asia-Pacific.

[22]  Ioan Stefan Sacala,et al.  Generic Architecture for Process Mining in the Context of Cyber Physical Systems , 2014 .

[23]  Thomas Kirste,et al.  Computational causal behaviour models for assisted manufacturing , 2015, iWOAR.

[24]  Luc Van Gool,et al.  Tools for semi-automatic monitoring of industrial workflows , 2010, ARTEMIS '10.

[25]  Theodora A. Varvarigou,et al.  A Threefold Dataset for Activity and Workflow Recognition in Complex Industrial Environments , 2012, IEEE MultiMedia.

[26]  Bala Srinivasan,et al.  Activity Recognition with Evolving Data Streams , 2018, ACM Comput. Surv..

[27]  Gernot A. Fink,et al.  Deep Neural Network based Human Activity Recognition for the Order Picking Process , 2017, iWOAR.

[28]  Paul Lukowicz,et al.  Wearable Activity Tracking in Car Manufacturing , 2008, IEEE Pervasive Computing.

[29]  Andreas Dengel,et al.  Hierarchical Model for Zero-shot Activity Recognition using Wearable Sensors , 2018, ICAART.

[30]  Paul Lukowicz,et al.  Continuous activity recognition in a maintenance scenario: combining motion sensors and ultrasonic hands tracking , 2012, Pattern Analysis and Applications.

[31]  Paul Lukowicz,et al.  Distributed Modular Toolbox for Multi-modal Context Recognition , 2006, ARCS.

[32]  Nirmalya Roy,et al.  Recent trends in machine learning for human activity recognition—A survey , 2018, WIREs Data Mining Knowl. Discov..

[33]  S AbdallahZahraa,et al.  Activity Recognition with Evolving Data Streams , 2018 .

[34]  Takuya Maekawa,et al.  Toward practical factory activity recognition: unsupervised understanding of repetitive assembly work in a factory , 2016, UbiComp.

[35]  Cassandra Mariette Carley.  Human Activity Analysis , 2018 .

[36]  Paul J. M. Havinga,et al.  Distributed Activity Recognition with Fuzzy-Enabled Wireless Sensor Networks , 2007, DCOSS.

[37]  Mathias Weske,et al.  The Internet-of-Things Meets Business Process Management: Mutual Benefits and Challenges , 2017, ArXiv.

[38]  Miguel A. Labrador,et al.  A Survey on Human Activity Recognition using Wearable Sensors , 2013, IEEE Communications Surveys & Tutorials.

[39]  Gernot A. Fink,et al.  Motion Classification for Analyzing the Order Picking Process using Mobile Sensors - General Concepts, Case Studies and Empirical Evaluation , 2016, ICPRAM.

[40]  J.K. Aggarwal,et al.  Human activity analysis , 2011, ACM Comput. Surv..

[41]  Wil M. P. van der Aalst,et al.  Process Mining , 2016, Springer Berlin Heidelberg.

[42]  Paul Lukowicz,et al.  Using a complex multi-modal on-body sensor system for activity spotting , 2008, 2008 12th IEEE International Symposium on Wearable Computers.

[43]  Alois Knoll,et al.  Multimodal Human Activity Recognition for Industrial Manufacturing Processes in Robotic Workcells , 2015, ICMI.