Error detection through mouse movement in an online adaptive learning environment

While response time and accuracy indicate overall performance, their value in uncovering cognitive processes, underlying learning, is limited. A promising online measure, designed to track decision-making, is computer mouse tracking, where mouse attraction towards different locations may reflect the consideration of alternative response options. Using a speedy arithmetic multiple-choice game in an online adaptive learning environment, we examined whether mouse movements could reflect arithmetic difficulties when error rates are low. Results showed that mouse movements towards alternative responses in correctly answered questions mapped onto the frequency of errors made in this online learning system. This mapping was stronger for the younger children, as well as for easy arithmetic problems. On an individual level, users showed more mouse movement towards their previously made response errors than towards other alternative options. This opens the possibility of adapting feedback and instruction on an individual basis through mouse tracking.

[1]  John Seely Brown,et al.  Diagnostic Models for Procedural Bugs in Basic Mathematical Skills , 1978, Cogn. Sci..

[2]  Jamie I. D. Campbell Network interference and mental multiplication. , 1987 .

[3]  R. Siegler,et al.  Strategy choice procedures and the development of multiplication skill. , 1988, Journal of experimental psychology. General.

[4]  Gerald L. Lohse,et al.  Organizational Behavior and Human Decision Processes a Comparison of Two Process Tracing Methods for Choice Tasks , 2022 .

[5]  Gerald L. Lohse,et al.  A Comparison of Two Process Tracing Methods for Choice Tasks , 1996 .

[6]  Talia Ben-Zeev,et al.  Rational Errors and the Mathematical Mind , 1998 .

[7]  P. Federico Learning styles and student attitudes toward various aspects of network-based instruction , 2000 .

[8]  Brenda R. J. Jansen,et al.  The development of children's rule use on the balance scale task. , 2002, Journal of experimental child psychology.

[9]  Michael J. Spivey,et al.  The Continuity Of Mind , 2008 .

[10]  Michael J. Spivey,et al.  Continuous attraction toward phonological competitors. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[11]  Frank Domahs,et al.  What makes multiplication facts difficult. Problem size or neighborhood consistency? , 2006, Experimental psychology.

[12]  Michael J. Spivey,et al.  Continuous Dynamics in Real-Time Cognition , 2006 .

[13]  Michael J. Spivey,et al.  Graded motor responses in the time course of categorizing atypical exemplars , 2007, Memory & cognition.

[14]  K. Nakayama,et al.  Hidden cognitive states revealed in choice reaching tasks , 2009, Trends in Cognitive Sciences.

[15]  Jonathan B Freeman,et al.  MouseTracker: Software for studying real-time mental processing using a computer mouse-tracking method , 2010, Behavior research methods.

[16]  Eugene Agichtein,et al.  Towards predicting web searcher gaze position from mouse movements , 2010, CHI Extended Abstracts.

[17]  Thomas A. Farmer,et al.  Hand in Motion Reveals Mind in Motion , 2011, Front. Psychology.

[18]  Tom Verguts,et al.  Distance in Motion: Response Trajectories Reveal the Dynamics of Number Comparison , 2011, PloS one.

[19]  Joseph G. Johnson,et al.  Applying the decision moving window to risky choice: Comparison of eye-tracking and mouse-tracing methods , 2011, Judgment and Decision Making.

[20]  S. Klinkenberg,et al.  Computer adaptive practice of Maths ability using a new item response model for on the fly ability and difficulty estimation , 2011, Comput. Educ..

[21]  Ryen W. White,et al.  User see, user point: gaze and cursor alignment in web search , 2012, CHI.

[22]  G. Maris,et al.  Speed-Accuracy Response Models: Scoring Rules based on Response Time and Accuracy , 2012 .

[23]  Joseph G. Johnson,et al.  The response dynamics of preferential choice , 2013, Cognitive Psychology.

[24]  Stanislas Dehaene,et al.  How do we convert a number into a finger trajectory? , 2013, Cognition.

[25]  Silvia Wen-Yu Lee,et al.  A review of using eye-tracking technology in exploring learning from 2000 to 2012 , 2013 .

[26]  Brenda R. J. Jansen,et al.  The Influence of Experiencing Success in Math on Math Anxiety, Perceived Math Competence, and Math Performance. , 2013 .

[27]  M. Straatemeier,et al.  Math Garden: A new educational and scientific instrument , 2014 .

[28]  Martin H. Fischer,et al.  Pushing forward in embodied cognition: may we mouse the mathematical mind? , 2014, Front. Psychol..

[29]  Tyler Marghetis,et al.  The Quarterly Journal of Experimental Psychology Doing Arithmetic by Hand: Hand Movements during Exact Arithmetic Reveal Systematic, Dynamic Spatial Processing , 2022 .

[30]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[31]  N. Ambady,et al.  Investigating the Early Stages of Person Perception: The Asymmetry of Social Categorization by Sex vs. Age , 2014, PloS one.

[32]  Thomas J Faulkenberry Hand movements reflect competitive processing in numerical cognition. , 2014, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[33]  M. Straatemeier,et al.  Learning multiplication: An integrated analysis of the multiplication ability of primary school children and the difficulty of single digit and multidigit multiplication problems , 2015 .

[34]  J. Freeman,et al.  Advanced mouse-tracking analytic techniques for enhancing psychological science , 2015 .

[35]  Thomas J. Faulkenberry,et al.  Response trajectories capture the continuous dynamics of the size congruity effect. , 2016, Acta psychologica.

[36]  Korbinian Möller,et al.  EEG-based prediction of cognitive workload induced by arithmetic: a step towards online adaptation in numerical learning , 2016 .

[37]  Roland Langrock,et al.  moveHMM: an R package for the statistical modelling of animal movement data using hidden Markov models , 2016 .

[38]  Patrick Lemaire,et al.  What does EEG tell us about arithmetic strategies? A review. , 2016, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[39]  Trudy Buwalda,et al.  Explaining Mistakes in Single Digit Multiplication : A Cognitive Model , 2016 .

[40]  Martial Mermillod,et al.  Differential effects of visual uncertainty and contextual guidance on perceptual decisions: Evidence from eye and mouse tracking in visual search. , 2016, Journal of vision.

[41]  Felix Henninger,et al.  Mousetrap: An integrated, open-source mouse-tracking package , 2017, Behavior Research Methods.

[42]  Abe D. Hofman,et al.  Fast and slow strategies in multiplication , 2018, Learning and Individual Differences.

[43]  Thomas J. Faulkenberry,et al.  Tracking the Continuous Dynamics of Numerical Processing: A Brief Review and Editorial , 2018, J. Numer. Cogn..

[44]  D. McLean,et al.  trajr: An R package for characterisation of animal trajectories , 2018 .

[45]  Paul E. Stillman,et al.  How Mouse-tracking Can Advance Social Cognitive Theory , 2018, Trends in Cognitive Sciences.

[46]  Jonathan B. Freeman,et al.  Doing Psychological Science by Hand , 2018, Current directions in psychological science.

[47]  Matthew G. Huebner,et al.  Selection of Procedures in Mental Subtraction: Use of Eye Movements as a Window on Arithmetic Processing , 2017, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[48]  Abe D. Hofman,et al.  A Solution to the Measurement Problem in the Idiographic Approach Using Computer Adaptive Practicing , 2018, Journal of Intelligence.

[49]  G. Maris,et al.  Tracing Systematic Errors to Personalize Recommendations in Single Digit Multiplication and Beyond , 2018 .

[50]  Christopher D. Erb,et al.  The developing mind in action: measuring manual dynamics in childhood , 2018 .

[51]  Abe D. Hofman,et al.  The dynamics of the development of mathematics skills: A comparison of theories of developing intelligence. , 2018 .

[52]  Stefan Scherbaum,et al.  Stuck at the starting line: How the starting procedure influences mouse-tracking data , 2017, Behavior Research Methods.

[53]  Ann-Christine Ehlis,et al.  Individual Differences in Math Ability Determine Neurocognitive Processing of Arithmetic Complexity: A Combined fNIRS-EEG Study , 2019, Front. Hum. Neurosci..

[54]  Abe D. Hofman,et al.  Modeling person-specific development of math skills in continuous time: New evidence for mutualism , 2019, EDM.

[55]  Iroise Dumontheil,et al.  Should online math learning environments be tailored to individuals' cognitive profiles? , 2019, Journal of experimental child psychology.

[56]  Joseph G. Johnson,et al.  A Handbook of Process Tracing Methods , 2019 .

[57]  Fabio Babiloni,et al.  Brain–Computer Interface-Based Adaptive Automation to Prevent Out-Of-The-Loop Phenomenon in Air Traffic Controllers Dealing With Highly Automated Systems , 2019, Front. Hum. Neurosci..

[58]  Review for "Error detection through mouse movement in an online adaptive learning environment" , 2020 .