Integrated real-time, non-intrusive Measurements for Mental Load

In this position paper, we propose to develop a system that takes input data from different sensors on physiological behaviors such as Pupil Diameter, Blinking Rate, Heart Rate, Heart Rate Variability, and Galvanic Skin Response to estimate users’ mental load (see Sidebar 1). We firstly aim to collect data on these behaviours and then intend to understand the correlation between them and later want to predict an estimate of cognitive load in real-time. We hope to use this measure during Human-Computer Interaction and Human-Robot Interaction to personalize our interfaces or robots’ behaviour according to user mental load in the real-time.

[1]  Jamie Reilly,et al.  The human task-evoked pupillary response function is linear: Implications for baseline response scaling in pupillometry , 2018, Behavior Research Methods.

[2]  Antonio Luque-Casado,et al.  Cognitive Performance and Heart Rate Variability: The Influence of Fitness Level , 2013, PloS one.

[3]  F. Paas,et al.  Cognitive Load Measurement as a Means to Advance Cognitive Load Theory , 2003 .

[4]  B. Oken,et al.  Sensitivity to mental effort and test–retest reliability of heart rate variability measures in healthy seniors , 2011, Clinical Neurophysiology.

[5]  Katrin S. Lohan,et al.  Playing Pairs with Pepper , 2018, HRI 2018.

[6]  Jan Cech,et al.  Real-Time Eye Blink Detection using Facial Landmarks , 2016 .

[7]  M. Holland,et al.  Blinking and Mental Load , 1972, Psychological reports.

[8]  Andrew L. Kun,et al.  Exploring the effects of size and luminance of visual targets on the pupillary light reflex , 2012, AutomotiveUI.

[9]  J. Sweller.  COGNITIVE LOAD THEORY, LEARNING DIFFICULTY, AND INSTRUCTIONAL DESIGN , 1994 .

[10]  James A. Larson,et al.  Guidelines for multimodal user interface design , 2004, CACM.

[11]  A. Bentivoglio,et al.  Analysis of blink rate patterns in normal subjects , 1997, Movement disorders : official journal of the Movement Disorder Society.

[12]  Yang Wang,et al.  Multimodal behavior and interaction as indicators of cognitive load , 2012, TIIS.

[13]  S. Steinhauer,et al.  Blink before and after you think: blinks occur prior to and following cognitive load indexed by pupillary responses. , 2008, Psychophysiology.

[14]  Katrin S. Lohan,et al.  Using Pupil Diameter to Measure Cognitive Load , 2018, HRI 2018.

[15]  F. Paas,et al.  Memory load and the cognitive pupillary response in aging. , 2004, Psychophysiology.

[16]  Frank Broz,et al.  Gaze and Attention During an HRI Storytelling Task , 2015, HRI 2015.

[17]  F. Paas,et al.  Cognitive Architecture and Instructional Design , 1998 .

[18]  Omar Mubin,et al.  A Systematic Review of Adaptivity in Human-Robot Interaction , 2017, Multimodal Technol. Interact..

[19]  Nathaniel P. Grove,et al.  Measuring Load on Working Memory: The Use of Heart Rate as a Means of Measuring Chemistry Students' Cognitive Load , 2014 .