Towards Multimodal Office Task Performance Estimation

The performance of human workers can be fluctuated due to changes in the cognitive state during sustained work. Though past researches have made human performance monitoring possible by utilizing physiological signals, little attention has been paid to the context of office works. This research proposes a multimodal approach to estimate office task performance. A transcription typing experiment was conducted to simulate the real working environment while typing speed and error rate represented as performance metrics. Physiological data collected during the experiment, together with conventional machine learning algorithms showed feasibility to accurately predict two levels (good/bad) of task performance. More importantly, a comprehensive comparison between choices of modality suggests that using data from particular sources could gain predictive performance comparable to the complete set of modalities.

[1]  Harri Haapasalo,et al.  Productivity and Performance Management – Managerial Practices in the Construction Industry , 2012 .

[2]  Michael E. Smith,et al.  Monitoring Task Loading with Multivariate EEG Measures during Complex Forms of Human-Computer Interaction , 2001, Hum. Factors.

[3]  G. Borghini,et al.  Neuroscience and Biobehavioral Reviews , 2022 .

[4]  Christopher A. Higgins,et al.  Computerized performance monitoring systems: use and abuse , 1986, CACM.

[5]  Matthias Beggiato,et al.  Using Smartbands, Pupillometry and Body Motion to Detect Discomfort in Automated Driving , 2018, Front. Hum. Neurosci..

[6]  Michael E. Smith,et al.  Neurophysiological measures of cognitive workload during human-computer interaction , 2003 .

[7]  Bryan Reimer,et al.  Classifying driver workload using physiological and driving performance data: two field studies , 2014, CHI.

[8]  Helen Petrie,et al.  Can Listening to Music Make You Type Better? The Effect of Music Style, Vocals and Volume on Typing Performance , 2016, ICAD 2016.

[9]  Gerd Wanielik,et al.  Head tracking based glance area estimation for driver behaviour modelling during lane change execution , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[10]  Luca Citi,et al.  cvxEDA: A Convex Optimization Approach to Electrodermal Activity Processing , 2016, IEEE Transactions on Biomedical Engineering.

[11]  Vincent G. Duffy,et al.  Development of a facial skin temperature-based methodology for non-intrusive mental workload measurement , 2007 .

[12]  Sebastiaan Overeem,et al.  Manipulation of core body and skin temperature improves vigilance and maintenance of wakefulness in narcolepsy. , 2008, Sleep.

[13]  José Neves,et al.  A multi-modal architecture for non-intrusive analysis of performance in the workplace , 2017, Neurocomputing.

[14]  T. Sejnowski,et al.  Removing electroencephalographic artifacts by blind source separation. , 2000, Psychophysiology.

[15]  T. Higuchi.  Approach to an irregular time series on the basis of the fractal theory , 1988 .

[16]  Amirhossein H. Memar,et al.  Physiological Measures for Human Performance Analysis in Human-Robot Teamwork: Case of Tele-Exploration , 2018, IEEE Access.

[17]  J. Martinerie,et al.  Prediction of performance level during a cognitive task from ongoing EEG oscillatory activities , 2008, Clinical Neurophysiology.

[18]  C. Colleta,et al.  Autonomic nervous system and subjective ratings of strain in air-traffic control , 2008 .

[19]  G. Ben-Shakhar,et al.  Standardization within individuals: a simple method to neutralize individual differences in skin conductance. , 1985, Psychophysiology.

[20]  Shutaro Kunimasa,et al.  An Estimation Method of Intellectual Work Performance by Using Physiological Indices , 2019, Transactions of the Society of Instrument and Control Engineers.

[21]  R. Simons,et al.  To err is autonomic: error-related brain potentials, ANS activity, and post-error compensatory behavior. , 2003, Psychophysiology.

[22]  Martin Luessi,et al.  MNE software for processing MEG and EEG data , 2014, NeuroImage.

[23]  Jennifer Healey,et al.  Toward Machine Emotional Intelligence: Analysis of Affective Physiological State , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[25]  I. Heynderickx,et al.  Lighting up the office: The effect of wall luminance on room appraisal, office workers' performance, and subjective alertness , 2018, Building and Environment.