Exploring the Usage of Commercial Bio-Sensors for Multitasking Detection

Most of the current adaptive systems support single task activities. The rise in the number of daily interactive devices and sources of information made multitasking an integral activity in our daily life. Affect-aware systems show exciting potential to support the user, however, they focus on the induced effect of an additional task in terms of cognitive load and stress, rather than the influence of the number of tasks i.e. multitasking. This paper presents indicators of the number of tasks being performed by the user using a set of bio-sensors. A preliminary user study was conducted with two follow-up explorations. Our findings imply that we can distinguish between the number of tasks performed based on high-end as well as cheap Heart Rate sensors. Additionally, tasks number correlates with other signals, namely wrist and forehead temperature. We provide empirical evidence showing how to differentiate between single- and dual-tasking activities.

[1]  U. Rajendra Acharya,et al.  Heart rate variability: a review , 2006, Medical and Biological Engineering and Computing.

[2]  Ricardo Gutierrez-Osuna,et al.  Using Heart Rate Monitors to Detect Mental Stress , 2009, 2009 Sixth International Workshop on Wearable and Implantable Body Sensor Networks.

[3]  Federica Scarpina,et al.  The Stroop Color and Word Test , 2017, Front. Psychol..

[4]  Markus Funk,et al.  ABBAS: An Adaptive Bio-sensors Based Assistive System , 2017, CHI Extended Abstracts.

[5]  S. Huffel,et al.  Influence of Mental Stress on Heart Rate and Heart Rate Variability , 2009 .

[6]  Martin L. Griss,et al.  Activity-Aware Mental Stress Detection Using Physiological Sensors , 2010, MobiCASE.

[7]  Xue-Wei Tian,et al.  A fatigue detection algorithm by heart rate variability based on a neuro-fuzzy network , 2011, ICUIMC '11.

[8]  Frédéric Bousefsaf,et al.  Remote detection of mental workload changes using cardiac parameters assessed with a low-cost webcam , 2014, Comput. Biol. Medicine.

[9]  D. Adam,et al.  Assessment of autonomic function in humans by heart rate spectral analysis. , 1985, The American journal of physiology.

[10]  P. Melillo,et al.  Nonlinear Heart Rate Variability features for real-life stress detection. Case study: students under stress due to university examination , 2011, Biomedical engineering online.

[11]  V. Somers,et al.  Heart Rate Variability: , 2003, Journal of cardiovascular electrophysiology.

[12]  Roberto Colom,et al.  Intelligence, working memory, and multitasking performance , 2010 .

[13]  Mykola Pechenizkiy,et al.  What's Your Current Stress Level? Detection of Stress Patterns from GSR Sensor Data , 2011, 2011 IEEE 11th International Conference on Data Mining Workshops.

[14]  Julia Wache,et al.  The Secret Language of Our Body: Affect and Personality Recognition Using Physiological Signals , 2014, ICMI.

[15]  R. Pea.  User Centered System Design: New Perspectives on Human-Computer Interaction , 1987 .

[16]  Niels Henze,et al.  See through the fire: evaluating the augmentation of visual perception of firefighters using depth and thermal cameras , 2017, UbiComp/ISWC Adjunct.

[17]  Veikko Surakka,et al.  Emotions and heart rate while sitting on a chair , 2005, CHI.

[18]  Ioannis Pavlidis,et al.  Contact-free Stress Monitoring for User's Divided Attention , 2008 .

[19]  Adrian Burns,et al.  SHIMMER™ – A Wireless Sensor Platform for Noninvasive Biomedical Research , 2010, IEEE Sensors Journal.

[20]  P. Stein,et al.  Heart Rate Variability: Measurement and Clinical Utility , 2005, Annals of noninvasive electrocardiology : the official journal of the International Society for Holter and Noninvasive Electrocardiology, Inc.

[21]  Alireza Sahami Shirazi,et al.  Smarttention, please!: 2nd workshop on intelligent attention management on mobile devices , 2016, MobileHCI Adjunct.

[22]  István Vassányi,et al.  Stress Detection Using Low Cost Heart Rate Sensors , 2016, Journal of healthcare engineering.

[23]  H. Huikuri,et al.  Time domain, geometrical and frequency domain analysis of cardiac vagal outflow: effects of various respiratory patterns. , 2001, Clinical physiology.

[24]  Markus Funk,et al.  One size does not fit all: challenges of providing interactive worker assistance in industrial settings , 2017, UbiComp/ISWC Adjunct.

[25]  Matthias Scheutz,et al.  Sensing cognitive multitasking for a brain-based adaptive user interface , 2011, CHI.

[26]  T. Shallice,et al.  The cognitive and neuroanatomical correlates of multitasking , 2000, Neuropsychologia.

[27]  Gari D. Clifford,et al.  Signal processing methods for heart rate variability , 2002 .

[28]  Yang Wang,et al.  Using galvanic skin response for cognitive load measurement in arithmetic and reading tasks , 2012, OZCHI.

[29]  H. Prendinger,et al.  Emotion Recognition from Electromyography and Skin Conductance , 2005 .

[30]  E. D. de Geus,et al.  Effects of work stress on ambulatory blood pressure, heart rate, and heart rate variability. , 2000, Hypertension.

[31]  Janko Drnovsek,et al.  Non-contact heart rate and heart rate variability measurements: A review , 2014, Biomed. Signal Process. Control..

[32]  P. Lang.  The emotion probe. Studies of motivation and attention. , 1995, The American psychologist.

[33]  Markus Funk,et al.  Stop helping me - I'm bored!: why assembly assistance needs to be adaptive , 2015, UbiComp/ISWC Adjunct.

[34]  Ioannis T. Pavlidis,et al.  StressCam: non-contact measurement of users' emotional states through thermal imaging , 2005, CHI Extended Abstracts.

[35]  Shiv Naresh Shivhare,et al.  Emotion Detection from Text , 2012, ArXiv.

[36]  Florian Alt,et al.  Stay Cool! Understanding Thermal Attacks on Mobile-based User Authentication , 2017, CHI.

[37]  Niels Henze,et al.  UbiTtention: smart & ambient notification and attention management , 2016, UbiComp Adjunct.

[38]  Donald A. Norman,et al.  User Centered System Design: New Perspectives on Human-Computer Interaction , 1988 .

[39]  N. Shah,et al.  Haemodynamic parameters in hypertensive patients following acute stroke and early outcome , 2008 .

[40]  Daniel McDuff,et al.  Remote measurement of cognitive stress via heart rate variability , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[41]  Fang Chen,et al.  Galvanic skin response (GSR) as an index of cognitive load , 2007, CHI Extended Abstracts.

[42]  Alireza Sahami Shirazi,et al.  Investigation of Material Properties for Thermal Imaging-Based Interaction , 2015, CHI.

[43]  Simon Ollander.  Wearable Sensor Data Fusion for Human Stress Estimation , 2015 .

[44]  George N. Votsis,et al.  Emotion recognition in human-computer interaction , 2001, IEEE Signal Process. Mag..

[45]  Janne Heikkilä,et al.  Predicting the Valence of a Scene from Observers’ Eye Movements , 2015, PloS one.

[46]  Slim Abdennadher,et al.  DiVA: exploring the usage of pupil diameter to elicit valence and arousal , 2017, MUM.

[47]  Fang Chen,et al.  Using Galvanic Skin Response (GSR) to Measure Trust and Cognitive Load in the Text-Chat Environment , 2015, CHI Extended Abstracts.

[48]  J. Lagopoulos.  Electrodermal activity , 2007, Acta Neuropsychiatrica.

[49]  Niels Henze,et al.  Screen arrangements and interaction areas for large display work places , 2016, PerDis.

[50]  Egon L. van den Broek,et al.  Ubiquitous emotion-aware computing , 2011, Personal and Ubiquitous Computing.

[51]  A. Malliani,et al.  Heart rate variability. Standards of measurement, physiological interpretation, and clinical use , 1996 .

[52]  M. A. Rowe,et al.  Guide for Analysing Electrodermal Activity & Skin Conductance Responses for Psychological Experiments , 2013 .

[53]  Samia Nefti-Meziani,et al.  Predicting the Valence of a Scene from Observers’ Eye Movements , 2015, PloS one.

[54]  Albrecht Schmidt,et al.  Cognitive Heat: Exploring the Usage of Thermal Imaging to Unobtrusively Estimate Cognitive Load , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[55]  W. Boucsein.  Electrodermal activity, 2nd ed. , 2012 .