Prediction of Users' Learning Curves for Adaptation while Using an Information Visualization

User performance and satisfaction when working with an interface is influenced by how quickly the user can acquire the skills necessary to work with the interface through practice. Learning curves are mathematical models that can represent a user's skill acquisition ability through parameters that describe the user's initial expertise as well as her learning rate. This information could be used by an interface to provide adaptive support to users who may otherwise be slow in learning the necessary skills. In this paper, we investigate the feasibility of predicting in real time a user's learning curve when working with ValueChart, an interactive visualization for decision making. Our models leverage various data sources (a user's gaze behavior, pupil dilation, cognitive abilities), and we show that they outperform a baseline that leverages only knowledge on user task performance so far. We also show that the best performing model achieves good accuracies in predicting users' learning curves even after observing users' performance only on a few tasks. These results are promising toward the design of user-adaptive visualizations that can dynamically support a user in acquiring the necessary skills to complete visual tasks.

[1]  Sidney K. D'Mello,et al.  Evaluation of a Personalized Method for Proactive Mind Wandering Reduction , 2014, UMAP Workshops.

[2]  Cristina Conati,et al.  Towards facilitating user skill acquisition: identifying untrained visualization users through eye tracking , 2014, IUI.

[3]  E. Hess,et al.  Pupil Size in Relation to Mental Activity during Simple Problem-Solving , 1964, Science.

[4]  Giuseppe Carenini,et al.  ValueCharts: analyzing linear models expressing preferences and evaluations , 2004, AVI.

[5]  Ian H. Witten,et al.  WEKA: a machine learning workbench , 1994, Proceedings of ANZIIS '94 - Australian New Zealnd Intelligent Information Systems Conference.

[6]  Giuseppe Carenini,et al.  A Decision Support System for the Design and Evaluation of Sustainable Wastewater Solutions , 2014, IEEE Transactions on Computers.

[7]  Ying Zhu,et al.  Measuring Effective Data Visualization , 2007, ISVC.

[8]  Joseph E. Beck,et al.  ADVISOR: A Machine Learning Architecture for Intelligent Tutor Construction , 2000, AAAI/IAAI.

[9]  Lorenzo Vigentini,et al.  Visualising Virtual Learning Environments: Case Studies of the Website Exploration Tool , 2010, 2010 14th International Conference Information Visualisation.

[10]  Andrew Olney,et al.  Gaze tutor: A gaze-reactive intelligent tutoring system , 2012, Int. J. Hum. Comput. Stud..

[11]  Cristina Conati,et al.  Comparing and Combining Eye Gaze and Interface Actions for Determining User Learning with an Interactive Simulation , 2013, UMAP.

[12]  Kenneth R. Koedinger,et al.  Individualized Bayesian Knowledge Tracing Models , 2013, AIED.

[13]  K. Kirsner,et al.  Beyond the Learning Curve: The Construction of Mind , 2005 .

[14]  Cristina Conati,et al.  Highlighting interventions and user differences: informing adaptive information visualization support , 2014, CHI.

[15]  W. Kruskal Relative Importance by Averaging Over Orderings , 1987 .

[16]  Michel C. Desmarais Performance comparison of item-to-item skills models with the IRT single latent trait model , 2011, UMAP'11.

[17]  Sébastien Lallé,et al.  Comparing Student Models in Different Formalisms by Predicting Their Impact on Help Success , 2013, AIED.

[18]  Arzu Çöltekin,et al.  Exploring the efficiency of users' visual analytics strategies based on sequence analysis of eye movement recordings , 2010, Int. J. Geogr. Inf. Sci..

[19]  Cristina Conati,et al.  Inferring Visualization Task Properties, User Performance, and User Cognitive Abilities from Eye Gaze Data , 2014, ACM Trans. Interact. Intell. Syst..

[20]  J. Rotter Generalized expectancies for internal versus external control of reinforcement. , 1966, Psychological monographs.

[21]  H. Akaike A new look at the statistical model identification , 1974 .

[22]  Roman Bednarik,et al.  A Computational Approach for Prediction of Problem-Solving Behavior Using Support Vector Machines and Eye-Tracking Data , 2013, Eye Gaze in Intelligent User Interfaces.

[23]  John T. Stasko,et al.  Low-level components of analytic activity in information visualization , 2005, IEEE Symposium on Information Visualization, 2005. INFOVIS 2005..

[24]  Arin Ghazarian,et al.  Automatic detection of users’ skill levels using high-frequency user interface events , 2010, User Modeling and User-Adapted Interaction.

[25]  Cristina Conati,et al.  Inferring Learning from Gaze Data during Interaction with an Environment to Support Self-Regulated Learning , 2013, AIED.

[26]  Cristina Conati,et al.  Evaluating the Impact of User Characteristics and Different Layouts on an Interactive Visualization for Decision Making , 2014, Comput. Graph. Forum.

[27]  Kenneth R. Koedinger,et al.  Performance Factors Analysis - A New Alternative to Knowledge Tracing , 2009, AIED.

[28]  Brian P. Bailey,et al.  Towards an index of opportunity: understanding changes in mental workload during task execution , 2004, CHI.

[29]  R. Engle,et al.  Is working memory capacity task dependent , 1989 .

[30]  Edward K. Vogel,et al.  The capacity of visual working memory for features and conjunctions , 1997, Nature.

[31]  E. Granholm,et al.  Pupillometric measures of cognitive and emotional processes. , 2004, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[32]  J. Beatty Task-evoked pupillary responses, processing load, and the structure of processing resources. , 1982 .

[33]  Minoru Nakayama,et al.  Attentive interfaces for users with disabilities: eye gaze for intention and uncertainty estimation , 2009, Universal Access in the Information Society.

[34]  P. Maeyer,et al.  Study of the attentive behavior of novice and expert map users using eye tracking , 2014 .

[35]  Cristina Conati,et al.  Predicting Affect from Gaze Data during Interaction with an Intelligent Tutoring System , 2014, Intelligent Tutoring Systems.

[36]  Chris North,et al.  An insight-based methodology for evaluating bioinformatics visualizations , 2005, IEEE Transactions on Visualization and Computer Graphics.

[37]  Cristina Conati,et al.  Supporting interface customization using a mixed-initiative approach , 2007, IUI '07.

[38]  John R. Anderson Cognitive Skills and Their Acquisition , 2013 .

[39]  Joseph E. Beck,et al.  Using Knowledge Tracing in a Noisy Environment to Measure Student Reading Proficiencies , 2006, Int. J. Artif. Intell. Educ..

[40]  Brian D. Fisher,et al.  Towards the Personal Equation of Interaction: The impact of personality factors on visual analytics interface interaction , 2010, 2010 IEEE Symposium on Visual Analytics Science and Technology.

[41]  Ruth B. Ekstrom,et al.  Manual for kit of factor-referenced cognitive tests , 1976 .

[42]  John R. Anderson,et al.  Rules of the Mind , 1993 .

[43]  Pascual Martínez-Gómez,et al.  Recognition of understanding level and language skill using measurements of reading behavior , 2014, IUI.

[44]  E. V. Mcintyre,et al.  The relation between fit and prediction for alternative forms of learning curves and relearning curves , 1997 .

[45]  Scott E. Hudson,et al.  Dynamic detection of novice vs. skilled use without a task model , 2007, CHI.