Cognitive State Measurement on Learning Materials by Utilizing Eye Tracker and Thermal Camera

We demonstrate how information derived from pervasive sensors can quantify cognitive states of learners while they are reading a textbook. Eye tracking is one of the most effective approaches to measuring reading behavior. For example, high fixation duration represents a reader's attention on a document. However, it is still a challenging task to predict the reason for the attention (i.e., is it because of his/her interest or trouble of understanding?). In this paper, we utilize additional sensing modality to solve the problem. On the dataset of 12 high school students' reading behaviors, we have found that the changing of pupil diameter and nose temperature are highly correlated with their cognitive states including their interests and efforts for reading/solving tasks on learning materials in Physics.

[1]  E. Deci,et al.  Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions. , 2000, Contemporary educational psychology.

[2]  Sazali Yaacob,et al.  Descriptive Analysis of Skin Temperature Variability of Sympathetic Nervous System Activity in Stress , 2012 .

[3]  Steffen Schaal,et al.  Theoretical concepts for using multimedia in science education , 2006 .

[4]  Kai Kunze,et al.  The eye as the window of the language ability: Estimation of English skills by analyzing eye movement while reading documents , 2015, 2015 13th International Conference on Document Analysis and Recognition (ICDAR).

[5]  Robert M. Sapolsky,et al.  Why Zebras Don't Get Ulcers , 1994 .

[6]  Kai Kunze,et al.  Towards Extraction of Subjective Reading Incomprehension: Analysis of Eye Gaze Features , 2015, CHI Extended Abstracts.

[7]  T. Chamorro‐Premuzic,et al.  The Hungry Mind , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[8]  E. Deci,et al.  Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. , 2000, The American psychologist.

[9]  Daniel Roggen,et al.  Recognition of visual memory recall processes using eye movement analysis , 2011, UbiComp '11.

[10]  S. Ainsworth.  DeFT: A Conceptual Framework for Considering Learning with Multiple Representations. , 2006 .

[11]  Andrew L. Kun,et al.  Estimating cognitive load using remote eye tracking in a driving simulator , 2010, ETRA.

[12]  K. Rayner.  Eye movements in reading and information processing: 20 years of research. , 1998, Psychological bulletin.

[13]  Andreas Dengel,et al.  Towards an intelligent textbook: eye gaze based attention extraction on materials for learning and instruction in physics , 2016, UbiComp Adjunct.

[14]  Kai Kunze,et al.  The wordometer 2.0: estimating the number of words you read in real life using commercial EOG glasses , 2016, UbiComp Adjunct.

[15]  Wolfgang Schnotz,et al.  The Cambridge Handbook of Multimedia Learning: An Integrated Model of Text and Picture Comprehension , 2005 .

[16]  P. Chandler,et al.  Cognitive Load Theory and the Format of Instruction , 1991 .

[17]  Kai Kunze,et al.  Position paper: brain teasers - toward wearable computing that engages our mind , 2014, UbiComp Adjunct.

[18]  Kai Kunze,et al.  The augmented narrative: toward estimating reader engagement , 2015, AH.

[19]  Kai Kunze,et al.  Reading Activity Recognition Using an Off-the-Shelf EEG -- Detecting Reading Activities and Distinguishing Genres of Documents , 2013, 2013 12th International Conference on Document Analysis and Recognition.

[20]  Andreas Dengel,et al.  The text 2.0 framework: writing web-based gaze-controlled realtime applications quickly and easily , 2010, EGIHMI '10.

[21]  Peter Robinson,et al.  Constrained Local Neural Fields for Robust Facial Landmark Detection in the Wild , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[22]  K. Rayner,et al.  Eye Movements as Reflections of Comprehension Processes in Reading , 2006 .

[23]  D Kahneman,et al.  Pupil Diameter and Load on Memory , 1966, Science.

[24]  Andreas Dengel,et al.  Eye movements as implicit relevance feedback , 2008, CHI Extended Abstracts.

[25]  Kai Kunze,et al.  Quantifying reading habits: counting how many words you read , 2015, UbiComp.

[26]  E. Hess,et al.  Pupil Size in Relation to Mental Activity during Simple Problem-Solving , 1964, Science.

[27]  Erik D. Reichle,et al.  Eye movements in reading and information processing : , 2015 .

[28]  Frederick F. Schmitt,et al.  The epistemic value of curiosity , 2008 .