A framework for operator – workstation interaction in Industry 4.0

We draw on cognitive and behavioural theories and on the artificial intelligence literature in order to propose a framework of future operator – workstation interaction in the ‘Industry 4.0’ era. We name the proposed framework ‘Operator – Workstation Interaction 4.0’. The latter’s capabilities permit an adaptive, ongoing interaction that aims to improve operator safety, performance, well-being, and satisfaction as well as the factory’s production measures. The framework is composed of three subsystems: (1) the observation subsystem which observes the operator and the processes occurring in the workstation, (2) the analysis subsystem which generates understanding and implications of the observations output, (3) the reaction subsystem which determines if and how to respond. The paper describes these elements and illustrate them using an example of a fatigued worker. The contributions, implications, and limitations of the proposed framework are discussed, and future research directions are presented.

[1]  Xiaoping Chen,et al.  Recognizing slow eye movement for driver fatigue detection with machine learning approach , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[2]  Kyung S. Park Human Reliability: Analysis, Prediction, and Prevention of Human Errors , 1986 .

[3]  Ron Van Houten,et al.  Automated measurement in applied behavior analysis: A review , 2013 .

[4]  Lifeng Zhou,et al.  Industry 4.0: Towards future industrial opportunities and challenges , 2015, 2015 12th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD).

[5]  Sergio Salmeron-Majadas,et al.  Filtering of Spontaneous and Low Intensity Emotions in Educational Contexts , 2015, AIED.

[6]  Zhu Wang,et al.  Enabling non-invasive and real-time human-machine interactions based on wireless sensing and fog computing , 2018, Personal and Ubiquitous Computing.

[7]  Barry Strauch,et al.  Investigating Human Error: Incidents, Accidents, and Complex Systems , 2002 .

[8]  Freimut Bodendorf,et al.  Investigating Flexibility as a Performance Dimension of a Manufacturing Value Modeling Methodology (MVMM): A Framework for Identifying Flexibility Types in Manufacturing Systems , 2017 .

[9]  Guang-Zhong Yang,et al.  The use of pervasive sensing for behaviour profiling - a survey , 2009, Pervasive Mob. Comput..

[10]  K. Stecke,et al.  The evolution of production systems from Industry 2.0 through Industry 4.0 , 2018, Int. J. Prod. Res..

[11]  CambriaErik,et al.  A review of affective computing , 2017 .

[12]  Yang Xiao,et al.  Bio-inspired visual attention in agile sensing for target detection , 2009, Int. J. Sens. Networks.

[13]  Gang Li,et al.  Detection of Driver Drowsiness Using Wavelet Analysis of Heart Rate Variability and a Support Vector Machine Classifier , 2013, Sensors.

[14]  Yaoyao Fiona Zhao,et al.  Enabling cognitive manufacturing through automated on-machine measurement planning and feedback , 2010, Adv. Eng. Informatics.

[15]  Erik Cambria,et al.  A review of affective computing: From unimodal analysis to multimodal fusion , 2017, Inf. Fusion.

[16]  Satoshi Nakamura,et al.  Lip movement synthesis from speech based on hidden Markov models , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[17]  WöllmerMartin,et al.  Emotion on the road , 2010 .

[18]  Jennifer Healey,et al.  Toward Machine Emotional Intelligence: Analysis of Affective Physiological State , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Erik Cambria,et al.  Affective Computing and Sentiment Analysis , 2016, IEEE Intelligent Systems.

[20]  Mathias Schmitt,et al.  Human-machine-interaction in the industry 4.0 era , 2014, 2014 12th IEEE International Conference on Industrial Informatics (INDIN).

[21]  C. Büchel,et al.  Temporal and Spatial Dynamics of Brain Structure Changes during Extensive Learning , 2006, The Journal of Neuroscience.

[22]  M. Eysenck Attention And Arousal, Cognition And Performance , 1982 .

[23]  Qijie Zhao,et al.  Eye moving behaviors identification for gaze tracking interaction , 2014, Journal on Multimodal User Interfaces.

[24]  Mona Omidyeganeh,et al.  Intelligent driver drowsiness detection through fusion of yawning and eye closure , 2011, 2011 IEEE International Conference on Virtual Environments, Human-Computer Interfaces and Measurement Systems Proceedings.

[25]  Manfred Hild,et al.  Informatics for cognitive robots , 2010, Adv. Eng. Informatics.

[26]  Víctor Peláez,et al.  An automatic data mining method to detect abnormal human behaviour using physical activity measurements , 2014, Pervasive Mob. Comput..

[27]  Bin Guo,et al.  WhozDriving: Abnormal Driving Trajectory Detection by Studying Multi-faceted Driving Behavior Features , 2016, BigCom.

[28]  Stavros J. Perantonis,et al.  Detecting abnormal human behaviour using multiple cameras , 2009, Signal Process..

[29]  Roberto Brunelli,et al.  Person identification using multiple cues , 1995, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  J. Malouff,et al.  Increasing Emotional Intelligence through Training: Current Status and Future Directions , 2013 .

[31]  Minjuan Wang,et al.  Affective e-Learning: Using "Emotional" Data to Improve Learning in Pervasive Learning Environment , 2009, J. Educ. Technol. Soc..

[32]  A. Jenness,et al.  The recognition of facial expressions of emotion. , 1932 .

[33]  E. Walker,et al.  Diagnostic and Statistical Manual of Mental Disorders , 2013 .

[34]  Minoru Asada,et al.  Efficient human-robot collaboration: When should a robot take initiative? , 2017, Int. J. Robotics Res..

[35]  Michael J. Lyons,et al.  Coding facial expressions with Gabor wavelets , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[36]  Adam M. Grant Does intrinsic motivation fuel the prosocial fire? Motivational synergy in predicting persistence, performance, and productivity. , 2008, The Journal of applied psychology.

[37]  Aurobinda Routray,et al.  Automatic facial expression recognition using features of salient facial patches , 2015, IEEE Transactions on Affective Computing.

[38]  F. Chan,et al.  The link between Industry 4.0 and lean manufacturing: mapping current research and establishing a research agenda , 2018, Int. J. Prod. Res..

[39]  Peter Xiaoping Liu,et al.  Visual gesture recognition for human-machine interface of robot teleoperation , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[40]  Digitalization of Manufacturing Execution Systems : the core technology for realizing future Smart Factories , 2017 .

[41]  Frank Dellaert,et al.  Recognizing emotion in speech , 1996, Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96.

[42]  Gordon D Logan,et al.  Journal of Experimental Psychology : General Prevention and Correction in Post-Error Performance : An Ounce of Prevention , a Pound of Cure , 2012 .

[43]  Luis Miguel Bergasa,et al.  DriveSafe: An app for alerting inattentive drivers and scoring driving behaviors , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[44]  Phillip L. Ackerman,et al.  Individual differences in work motivation: Further explorations of a trait framework. , 2000 .

[45]  Christian Montag,et al.  Primal emotional-affective expressive foundations of human facial expression , 2016 .

[46]  Stanley B. Gershwin,et al.  The future of manufacturing systems engineering , 2018, Int. J. Prod. Res..

[47]  Björn W. Schuller,et al.  Emotion on the Road - Necessity, Acceptance, and Feasibility of Affective Computing in the Car , 2010, Adv. Hum. Comput. Interact..

[48]  P. de Jonge,et al.  Detecting mental disorders in general hospitals by the SCL-8 scale. , 2004, Journal of psychosomatic research.

[49]  J. M. O. Pinto,et al.  Human-Machine Interface (HMI) scenario quantification performed by ATHEANA, A Technique for Human Error Analysis , 2015 .

[50]  Lauri Nummenmaa,et al.  Recognition of Facial Expressions of Emotion is Related to their Frequency in Everyday Life , 2014, Journal of Nonverbal Behavior.

[51]  Álvaro Segura,et al.  Sustainable and flexible industrial human machine interfaces to support adaptable applications in the Industry 4.0 paradigm , 2019, Int. J. Prod. Res..

[52]  Cain C T Clark,et al.  A Review of Emerging Analytical Techniques for Objective Physical Activity Measurement in Humans , 2016, Sports Medicine.

[53]  Carsten Wittenberg,et al.  Human-CPS Interaction - requirements and human-machine interaction methods for the Industry 4.0 , 2016 .

[54]  Carlo Noe,et al.  Literature review on the ‘Smart Factory’ concept using bibliometric tools , 2017, Int. J. Prod. Res..

[55]  Michele Lora,et al.  Validation of HMI applications for industrial smart display , 2017, 2017 IEEE International High Level Design Validation and Test Workshop (HLDVT).

[56]  P D Bamidis,et al.  Affective Medicine , 2010, Methods of Information in Medicine.

[57]  Arkady B. Zaslavsky,et al.  Context Aware Computing for The Internet of Things: A Survey , 2013, IEEE Communications Surveys & Tutorials.

[58]  Anupam Agrawal,et al.  Vision based hand gesture recognition for human computer interaction: a survey , 2012, Artificial Intelligence Review.

[59]  Roberto Revetria,et al.  Augmented and virtual reality applications in industrial systems: A qualitative review towards the industry 4.0 era , 2018 .

[60]  José L. Martínez Lastra,et al.  From artificial cognitive systems and open architectures to cognitive manufacturing systems , 2015, 2015 IEEE 13th International Conference on Industrial Informatics (INDIN).

[61]  Conrad S. Tucker,et al.  Machine learning classification of design team members' body language patterns for real time emotional state detection , 2015 .

[62]  Qinghua Hu,et al.  Kernelized Fuzzy Rough Sets Based Yawn Detection for Driver Fatigue Monitoring , 2011, Fundam. Informaticae.

[63]  Tieniu Tan,et al.  Affective Computing: A Review , 2005, ACII.

[64]  Christopher D. Wickens,et al.  Stages and Levels of Automation: An Integrated Meta-analysis , 2010 .

[65]  L. Murukesan,et al.  Machine learning approach for sudden cardiac arrest prediction based on optimal heart rate variability features , 2014 .

[66]  Johan Stahre,et al.  TOWARDS AN OPERATOR 4.0 TYPOLOGY: A HUMAN-CENTRIC PERSPECTIVE ON THE FOURTH INDUSTRIAL REVOLUTION TECHNOLOGIES , 2016 .

[67]  Gangfeng Tan,et al.  Driving Fatigue Detection based on Blink Frequency and Eyes Movement , 2017 .

[68]  M. Carter Diagnostic and Statistical Manual of Mental Disorders, 5th ed. , 2014 .