Predicting Confusion in Information Visualization from Eye Tracking and Interaction Data

Confusion has been found to hinder user experience with visualizations. If confusion could be predicted and resolved in real time, user experience and satisfaction would greatly improve. In this paper, we focus on predicting occurrences of confusion during the interaction with a visualization using eye tracking and mouse data. The data was collected during a user study with ValueChart, an interactive visualization to support preferential choices. We report very promising results based on Random Forest classifiers.

[1]  Ryan Shaun Joazeiro de Baker,et al.  Automatic Detection of Learning-Centered Affective States in the Wild , 2015, IUI.

[2]  Ben Shneiderman,et al.  Querying event sequences by exact match or similarity search: Design and empirical evaluation , 2012, Interact. Comput..

[3]  Dimitris Kanellopoulos,et al.  Handling imbalanced datasets: A review , 2006 .

[4]  Avar Pental,et al.  Patterns of Confusion: Using Mouse Logs to Predict User's Emotional State. , 2015 .

[5]  M. Sheelagh T. Carpendale,et al.  Personal Visualization and Personal Visual Analytics , 2015, IEEE Transactions on Visualization and Computer Graphics.

[6]  Arthur C. Graesser,et al.  Mind and Body: Dialogue and Posture for Affect Detection in Learning Environments , 2007, AIED.

[7]  Dennis Gamayunov,et al.  Adaptive Visualization Interface That Manages User's Cognitive Load Based on Interaction Characteristics , 2014, VINCI '14.

[8]  Avar Pentel,et al.  Patterns of Confusion: Using Mouse Logs to Predict User's Emotional State , 2015, UMAP Workshops.

[9]  Cristina Conati,et al.  Inferring Visualization Task Properties, User Performance, and User Cognitive Abilities from Eye Gaze Data , 2014, ACM Trans. Interact. Intell. Syst..

[10]  Ryen W. White,et al.  Understanding web browsing behaviors through Weibull analysis of dwell time , 2010, SIGIR.

[11]  Vincent Aleven,et al.  Sensor-free automated detection of affect in a Cognitive Tutor for Algebra , 2012, EDM.

[12]  E. Granholm,et al.  Pupillometric measures of cognitive and emotional processes. , 2004, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[13]  Ming-An Wu.  Inferring user cognitive abilities from eye-tracking data , 2015 .

[14]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[15]  Sidney K. D'Mello,et al.  Automatic Gaze-Based Detection of Mind Wandering with Metacognitive Awareness , 2015, UMAP.

[16]  Alan Kennedy,et al.  Book Review: Eye Tracking: A Comprehensive Guide to Methods and Measures , 2016, Quarterly journal of experimental psychology.

[17]  Kasia Muldner,et al.  "Yes!": Using Tutor and Sensor Data to Predict Moments of Delight during Instructional Activities , 2010, UMAP.

[18]  Cristina Conati,et al.  When to Adapt: Detecting User's Confusion During Visualization Processing , 2013, UMAP Workshops.

[19]  Arjan Kuijper,et al.  User Similarity and Deviation Analysis for Adaptive Visualizations , 2014, HCI.

[20]  Vincent Aleven,et al.  Towards Sensor-Free Affect Detection in Cognitive Tutor Algebra. , 2012, EDM 2012.

[21]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[22]  Sung-Hee Kim,et al.  How do People Make Sense of Unfamiliar Visualizations?: A Grounded Model of Novice's Information Visualization Sensemaking , 2016, IEEE Transactions on Visualization and Computer Graphics.

[23]  Ji Soo Yi.  Visualized decision making: development and application of information visualization techniques to improve decision quality of nursing home choice , 2008 .

[24]  Cristina Conati,et al.  Prediction of Users' Learning Curves for Adaptation while Using an Information Visualization , 2015, IUI.

[25]  Cristina Gena,et al.  Methods and techniques for the evaluation of user-adaptive systems , 2005, The Knowledge Engineering Review.

[26]  Aladdin Ayesh,et al.  Using Mouse and Keyboard Dynamics to Detect Cognitive Stress During Mental Arithmetic , 2015 .

[27]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[28]  Andy Liaw,et al.  Classification and Regression by randomForest , 2007 .

[29]  Cristina Conati,et al.  Predicting Affect from Gaze Data during Interaction with an Intelligent Tutoring System , 2014, Intelligent Tutoring Systems.

[30]  Minho Lee,et al.  Identification of human implicit visual search intention based on eye movement and pupillary analysis , 2014, User Modeling and User-Adapted Interaction.

[31]  Brian P. Bailey,et al.  Towards an index of opportunity: understanding changes in mental workload during task execution , 2004, CHI.

[32]  Max Kuhn,et al.  Building Predictive Models in R Using the caret Package , 2008 .

[33]  Sucheta Nadkarni,et al.  A Task-Based Model of Perceived Website Complexity , 2007, MIS Q..

[34]  Cristina Conati,et al.  Knowledge Elicitation Methods for Affect Modelling in Education , 2013, Int. J. Artif. Intell. Educ..

[35]  Cristina Conati,et al.  Evaluating the Impact of User Characteristics and Different Layouts on an Interactive Visualization for Decision Making , 2014, Comput. Graph. Forum.