Automatic Cognitive Load Classification Using High-Frequency Interaction Events: An Exploratory Study

There is still a challenge of creating an evaluation method which can not only unobtrusively collect data without supplement equipment but also objectively, quantitatively and in real-time evaluate cognitive load of user based the data. The study explores the possibility of using the features extracted from high-frequency interaction events to evaluate cognitive load to respond to the challenge. Specifically, back-propagation neural networks, along with two feature selection methods nBset and SFS, were used as the classifier and it was able to use a set of features to differentiate three cognitive load levels with an accuracy of 74.27%. The main contributions of the research are: 1 demonstrating the use of combining machine learning techniques and the HFI features in automatically evaluating cognitive load; 2 showing the potential of using the HFI features in discriminating different cognitive load when suitable classifier and features are adopted.

[1]  Fang Chen,et al.  Towards Automatic Cognitive Load Measurement from Speech Analysis , 2007, HCI.

[2]  Roland Brünken,et al.  Cognitive Load Theory: THEORY , 2010 .

[3]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[4]  David B. Kaber,et al.  Situation awareness implications of adaptive automation for information processing in an air traffic control-related task , 2006 .

[5]  F. Paas,et al.  Cognitive Architecture and Instructional Design , 1998 .

[6]  J. G. Hollands,et al.  Engineering Psychology and Human Performance , 1984 .

[7]  Glenn F. Wilson,et al.  Performance Enhancement in an Uninhabited Air Vehicle Task Using Psychophysiologically Determined Adaptive Aiding , 2007, Hum. Factors.

[8]  Scott E. Hudson,et al.  Dynamic detection of novice vs. skilled use without a task model , 2007, CHI.

[9]  Pavlo D. Antonenko,et al.  The influence of leads on cognitive load and learning in a hypertext environment , 2010, Comput. Hum. Behav..

[10]  Slava Kalyuga,et al.  Evaluating and Managing Cognitive Load in Games , 2009 .

[11]  F. Paas,et al.  Variability of Worked Examples and Transfer of Geometrical Problem-Solving Skills: A Cognitive-Load Approach , 1994 .

[12]  Mihaela Cocea,et al.  Log file analysis for disengagement detection in e-Learning environments , 2009, User Modeling and User-Adapted Interaction.

[13]  John L. Sibert,et al.  Heart rate variability: indicator of user state as an aid to human-computer interaction , 1998, CHI.

[14]  David B. Boles,et al.  Predicting Dual-Task Performance With the Multiple Resources Questionnaire (MRQ) , 2007, Hum. Factors.

[15]  Mariano-Florentino Cuéllar,et al.  'Securing' the Nation: Law, Politics, and Organization at the Federal Security Agency, 1939-1953 , 2006 .

[16]  R. Vanderberg,et al.  Which components of working memory are important in the writing process? , 2007 .

[17]  M. Angela Sasse,et al.  From doing to being: getting closer to the user experience , 2004, Interact. Comput..

[18]  David F. Redmiles,et al.  Extracting usability information from user interface events , 2000, CSUR.

[19]  Martin C. Maguire,et al.  Evaluating User-Computer Interaction: A Framework , 1993, Int. J. Man Mach. Stud..

[20]  Fang Chen,et al.  Automatic cognitive load detection from speech features , 2007, OZCHI '07.

[21]  R. H. Mogford,et al.  The Complexity Construct in Air Traffic Control: A Review and Synthesis of the Literature. , 1995 .

[22]  Tjerk de Greef,et al.  Adaptive Automation in a Naval Combat Management System , 2009, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[23]  Mica R. Endsley,et al.  The Out-of-the-Loop Performance Problem and Level of Control in Automation , 1995, Hum. Factors.

[24]  Geoffrey Underwood,et al.  Inspecting Pictures for Information to Verify a Sentence: Eye Movements in General Encoding and in Focused Search , 2004, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[25]  Tao Lin,et al.  Using multiple data sources to get closer insights into user cost and task performance , 2008, Interact. Comput..

[26]  B. Rouse William,et al.  Adaptive Aiding for Human/Computer Control , 1988 .

[27]  A. Kramer,et al.  Physiological metrics of mental workload: A review of recent progress , 1990, Multiple-task performance.

[28]  Panayiotis Zaphiris,et al.  A model of cognitive loads in massively multiplayer online role playing games , 2007, Interact. Comput..

[29]  Slava Kalyuga,et al.  Managing split-attention and redundancy in multimedia instruction , 1999 .

[30]  Slava Kalyuga,et al.  Enhancing Instructional Efficiency of Interactive E-learning Environments: A Cognitive Load Perspective , 2007 .

[31]  F. Paas,et al.  Measurement of Cognitive Load in Instructional Research , 1994, Perceptual and motor skills.

[32]  Fang Chen,et al.  Cognitive skills learning: pen input patterns in computer-based athlete training , 2010, ICMI-MLMI '10.

[33]  David F. Redmiles,et al.  Large-Scale Collection of Usage Data to Inform Design , 2001, INTERACT.

[34]  Willem-Paul Brinkman,et al.  Component-Specific Usability Testing , 2008, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[35]  Jennifer Healey,et al.  Toward Machine Emotional Intelligence: Analysis of Affective Physiological State , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Sharon L. Oviatt,et al.  Quiet interfaces that help students think , 2006, UIST.

[37]  Mark A. Neerincx,et al.  Adaptive Automation Based on an Object-Oriented Task Model: Implementation and Evaluation in a Realistic C2 Environment , 2010 .

[38]  Fang Chen,et al.  Investigating speech features and automatic measurement of cognitive load , 2008, 2008 IEEE 10th Workshop on Multimedia Signal Processing.

[39]  F. Paas,et al.  Cognitive Load Measurement as a Means to Advance Cognitive Load Theory , 2003 .

[40]  Arin Ghazarian,et al.  Automatic detection of users’ skill levels using high-frequency user interface events , 2010, User Modeling and User-Adapted Interaction.

[41]  Tjerk de Greef,et al.  Eye Movement as Indicators of Mental Workload to Trigger Adaptive Automation , 2009, HCI.

[42]  F. Paas,et al.  Instructional control of cognitive load in the training of complex cognitive tasks , 1994 .

[43]  Misha Pavel,et al.  Unobtrusive monitoring of computer interactions to detect cognitive status in elders , 2004, IEEE Transactions on Information Technology in Biomedicine.

[44]  Paul Ayres.  Impact of Reducing Intrinsic Cognitive Load on Learning in a Mathematical Domain. , 2006 .

[45]  Tjerk de Greef,et al.  A Closed-Loop Adaptive System for Command and Control , 2007, HCI.

[46]  Fang Chen,et al.  Freeform pen-input as evidence of cognitive load and expertise , 2011, ICMI '11.

[47]  Andrew Sears,et al.  Automated stress detection using keystroke and linguistic features: An exploratory study , 2009, Int. J. Hum. Comput. Stud..

[48]  Tao Lin,et al.  The effects of virtual characters on audiences' movie experience , 2010, Interact. Comput..

[49]  Christopher D. Wickens,et al.  The Structure of Attentional Resources , 1980 .

[50]  Glenn F. Wilson,et al.  Real-Time Assessment of Mental Workload Using Psychophysiological Measures and Artificial Neural Networks , 2003, Hum. Factors.

[51]  Willem-Paul Brinkman,et al.  Towards an empirical method of efficiency testing of system parts: A methodological study , 2007, Interact. Comput..

[52]  Fang Chen,et al.  Cognitive Load Measurement from User's Linguistic Speech Features for Adaptive Interaction Design , 2009, INTERACT.

[53]  Mustapha Mouloua,et al.  Effects of Adaptive Task Allocation on Monitoring of Automated Systems , 1996, Hum. Factors.

[54]  Fang Chen,et al.  Cognitive load evaluation of handwriting using stroke-level features , 2011, IUI '11.

[55]  Fang Chen,et al.  Using pen input features as indices of cognitive load , 2007, ICMI '07.

[56]  Sandhya Samarasinghe,et al.  Neural Networks for Applied Sciences and Engineering , 2006 .