Characterisation of mobile-device tasks by their associated cognitive load through EEG data processing

Abstract Interaction with mobile devices serves as a link to the cyber world and allows us to characterise user behaviours. The deep analysis of the interactions with the smartphone, aligned with the principles of the Internet of People, allow us to distinguish between normal and abnormal use. One of the multiple applications of this type of analysis will contribute to the early diagnosis of mild cognitive impairment, based on anomalies in the interaction. This work aims to take the first steps towards that ambitious goal: to determine the cognitive load required for different typical tasks with smartphones. By properly identifying which tasks require a higher cognitive load, we will be able to start studying metrics and indicators that contribute to the early diagnosis of cognitive pathologies. The analysis of cognitive load was carried out after an experiment with 26 users who performed 12 typical tasks with a mobile device while their brain activity was monitored through electroencephalography. The results identify that there are clearly tasks with a higher cognitive demand, with audio production and consumption being the most notable, which has been validated experimentally and statistically.

[1]  Xinbo Gao,et al.  Deep Convolutional Neural Networks for mental load classification based on EEG data , 2018, Pattern Recognit..

[2]  Francine Grodstein,et al.  How early can cognitive decline be detected? , 2011, BMJ : British Medical Journal.

[3]  P. Achermann,et al.  Low-frequency (<1Hz) oscillations in the human sleep electroencephalogram , 1997, Neuroscience.

[4]  Kenton O'Hara,et al.  Pre-Touch Sensing for Mobile Interaction , 2016, CHI.

[5]  J. Sweller.  COGNITIVE LOAD THEORY, LEARNING DIFFICULTY, AND INSTRUCTIONAL DESIGN , 1994 .

[6]  Fang Yuan Chang,et al.  Usability Evaluation of Eye Tracker-Based Smart Phone APP GUI Design , 2014 .

[7]  Thierry Dutoit,et al.  Performance of the Emotiv Epoc headset for P300-based applications , 2013, Biomedical engineering online.

[8]  Virpi Roto,et al.  Interaction in 4-second bursts: the fragmented nature of attentional resources in mobile HCI , 2005, CHI.

[9]  Yang Wang,et al.  Using galvanic skin response for cognitive load measurement in arithmetic and reading tasks , 2012, OZCHI.

[10]  Blanka Klimova,et al.  Mobile Phone Apps in the Management and Assessment of Mild Cognitive Impairment and/or Mild-to-Moderate Dementia: An Opinion Article on Recent Findings , 2017, Front. Hum. Neurosci..

[11]  José Bravo,et al.  Using a Communication Model to Collect Measurement Data through Mobile Devices , 2012, Sensors.

[12]  Luis Alejandro Andrade-Lotero.  Teoría de la carga cognitiva, diseño multimedia y aprendizaje: un estado del arte , 2012 .

[13]  Pavlo D. Antonenko,et al.  Using Electroencephalography to Measure Cognitive Load , 2010 .

[14]  mc schraefel,et al.  A Taxonomy of Gestures in Human Computer Interactions , 2005 .

[15]  Richard Mayer,et al.  Multimedia Learning , 2001, Visible Learning Guide to Student Achievement.

[16]  Marcos Souza,et al.  Adding Eye Tracking Data Collection to Smartphone Usability Evaluation: A Comparison Between Eye Tracking Processes and Traditional Techniques , 2018, Advances in Usability, User Experience and Assistive Technology.

[17]  Yaacob Sazali,et al.  Classification of human emotion from EEG using discrete wavelet transform , 2010 .

[18]  Huali Wang,et al.  Mild cognitive impairment: a concept useful for early detection and intervention of dementia , 2013, Shanghai archives of psychiatry.

[19]  Don Towsley,et al.  Evidence to support common application switching behaviour on smartphones , 2019, Royal Society Open Science.

[20]  Luis Rodriguez-Benitez,et al.  Computational EEG Analysis Techniques When Playing Video Games: A Systematic Review , 2018, UCAmI.

[21]  David Edmund Johannes Linden,et al.  Identifying Indicators of Smartphone Addiction Through User-App Interaction , 2019, Comput. Hum. Behav..

[22]  Alex Fridman,et al.  Cognitive Load Estimation in the Wild , 2018, CHI.

[23]  Naveen Kumar,et al.  Measurement of Cognitive Load in HCI Systems Using EEG Power Spectrum: An Experimental Study , 2015, IHCI.

[24]  Deborah Lupton,et al.  'It's like having a physician in your pocket!' A critical analysis of self-diagnosis smartphone apps. , 2015, Social science & medicine.

[25]  Kristian Lukander,et al.  Estimating Brain Load from the EEG , 2009, TheScientificWorldJournal.

[26]  John Sweller,et al.  Discussion of ‘emerging topics in cognitive load research: using learner and information characteristics in the design of powerful learning environments’ , 2006 .

[27]  Deniz Mertkan Gezgin,et al.  Understanding Patterns for Smartphone Addiction: Age, Sleep Duration, Social Network Use and Fear of Missing Out , 2018, Cypriot Journal of Educational Sciences.

[28]  Walter Schneider,et al.  Controlled and Automatic Human Information Processing: 1. Detection, Search, and Attention. , 1977 .

[29]  W. Klimesch.  EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis , 1999, Brain Research Reviews.

[30]  Jesús Fontecha,et al.  A Friendly Navigation-System Based on Points of Interest, Augmented Reality and Context-Awareness , 2012, UCAmI.

[31]  Luis Rodriguez-Benitez,et al.  eeglib: computational analysis of cognitive performance during the use of video games , 2019, Journal of Ambient Intelligence and Humanized Computing.

[32]  Steve Love,et al.  Understanding Mobile Human-Computer Interaction , 2005 .

[33]  Eric Salmon,et al.  Dissociation between controlled and automatic processes in the behavioral variant of fronto-temporal dementia. , 2010, Journal of Alzheimer's disease : JAD.

[34]  Miriam Reiner,et al.  Real Time EEG Based Measurements of Cognitive Load Indicates Mental States During Learning , 2017, EDM 2017.

[35]  Zhaohui Wu,et al.  Discovering different kinds of smartphone users through their application usage behaviors , 2016, UbiComp.

[36]  W. Kirchner.  Age differences in short-term retention of rapidly changing information. , 1958, Journal of experimental psychology.

[37]  F. Paas,et al.  Cognitive Load Measurement as a Means to Advance Cognitive Load Theory , 2003 .

[38]  Ariana E. Anderson,et al.  The Relationship of Cognitive Performance and the Theta-Alpha Power Ratio Is Age-Dependent: An EEG Study of Short Term Memory and Reasoning during Task and Resting-State in Healthy Young and Old Adults , 2017, Front. Aging Neurosci..

[39]  H. Jasper,et al.  The ten-twenty electrode system of the International Federation. The International Federation of Clinical Neurophysiology. , 1999, Electroencephalography and clinical neurophysiology. Supplement.

[40]  Yu Huang,et al.  Assessing social anxiety using gps trajectories and point-of-interest data , 2016, UbiComp.

[41]  Caroline P. C. Chanel,et al.  Online ECG-based Features for Cognitive Load Assessment , 2019, 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC).

[42]  H. Simon,et al.  Why are some problems hard? Evidence from Tower of Hanoi , 1985, Cognitive Psychology.

[43]  Jesús Fontecha,et al.  Achieving Adaptive Augmented Reality through Ontological Context-Awareness applied to AAL Scenarios , 2013, J. Univers. Comput. Sci..

[44]  Tina Seufert,et al.  Development and Validation of Two Instruments Measuring Intrinsic, Extraneous, and Germane Cognitive Load , 2017, Front. Psychol..

[45]  Jesús Favela,et al.  Assessing empathy and managing emotions through interactions with an affective avatar , 2018, Health Informatics J..

[46]  Jesús Fontecha,et al.  Analysis of Cognitive Load Using EEG when Interacting with Mobile Devices , 2019, UCAmI.

[47]  A. Paivio,et al.  Dual coding theory and education , 1991 .

[48]  Richard E. Mayer,et al.  e-Learning and the Science of Instruction: Proven Guidelines for Consumers and Designers of Multimedia Learning , 2002 .

[49]  Pauline van der Wel,et al.  Pupil dilation as an index of effort in cognitive control tasks: A review , 2018 .

[50]  Rui Wang,et al.  Using Smartphones to Collect Behavioral Data in Psychological Science , 2016, Perspectives on psychological science : a journal of the Association for Psychological Science.

[51]  Desney S. Tan,et al.  Feasibility and pragmatics of classifying working memory load with an electroencephalograph , 2008, CHI.

[52]  Young-Ju Lee,et al.  A Study on Content and Interaction Types on Smart-phone , 2015 .