Mouse tracking as a window into decision making

Mouse tracking promises to be an efficient method to investigate the dynamics of cognitive processes: It is easier to deploy than eyetracking, yet in principle it is much more fine-grained than looking at response times. We investigated these claimed benefits directly, asking how the features of decision processes—notably, decision changes—might be captured in mouse movements. We ran two experiments, one in which we explicitly manipulated whether our stimuli triggered a flip in decision, and one in which we replicated more ecological, classical mouse-tracking results on linguistic negation (Dale & Duran, Cognitive Science, 35, 983–996, 2011). We concluded, first, that spatial information (mouse path) is more important than temporal information (speed and acceleration) for detecting decision changes, and we offer a comparison of the sensitivities of various typical measures used in analyses of mouse tracking (area under the trajectory curve, direction flips, etc.). We do so using an “optimal” analysis of our data (a linear discriminant analysis explicitly trained to classify trajectories) and see what type of data (position, speed, or acceleration) it capitalizes on. We also quantify how its results compare with those based on more standard measures.

[1]  Michael J. Spivey,et al.  Tracking the Continuity of Language Comprehension: Computer Mouse Trajectories Suggest Parallel Syntactic Processing , 2007, Cogn. Sci..

[2]  Jarrod Moss,et al.  Mouse-tracking evidence for parallel anticipatory option evaluation , 2017, Cognitive Processing.

[3]  D. Wolpert,et al.  Changing your mind: a computational mechanism of vacillation , 2009, Nature.

[4]  P. Wason The contexts of plausible denial , 1965 .

[5]  Mante S. Nieuwland,et al.  When the Truth Is Not Too Hard to Handle , 2008, Psychological science.

[6]  James L. McClelland,et al.  The time course of perceptual choice: the leaky, competing accumulator model. , 2001, Psychological review.

[7]  Rick Dale,et al.  The Cognitive Dynamics of Negated Sentence Verification , 2011, Cogn. Sci..

[8]  C. Santamaría,et al.  How negation is understood: Evidence from the visual world paradigm , 2014 .

[9]  K. Nakayama,et al.  Hidden cognitive states revealed in choice reaching tasks , 2009, Trends in Cognitive Sciences.

[10]  F. Donders On the speed of mental processes. , 1969, Acta psychologica.

[11]  John M. Tomlinson,et al.  Possibly All of that and Then Some: Scalar Implicatures Are Understood in Two Steps. , 2013 .

[12]  Herbert H. Clark,et al.  On the process of comparing sentences against pictures , 1972 .

[13]  Uri Hasson,et al.  Does understanding negation entail affirmation?: An examination of negated metaphors , 2006 .

[14]  Takashi Yamauchi,et al.  Semantic priming revealed by mouse movement trajectories , 2014, Consciousness and Cognition.

[15]  P. Johnson-Laird,et al.  Psychology of Reasoning: Structure and Content , 1972 .

[16]  J. Franklin,et al.  The elements of statistical learning: data mining, inference and prediction , 2005 .

[17]  Takashi Yamauchi,et al.  The role of attention in subliminal semantic processing: A mouse tracking study , 2017, PloS one.

[18]  Rolf A. Zwaan,et al.  Experiential simulations of negated text information , 2007, Quarterly journal of experimental psychology.

[19]  C. Buss,et al.  Children's Brain Development Benefits from Longer Gestation , 2011, Front. Psychology.

[20]  Jonathan B Freeman,et al.  MouseTracker: Software for studying real-time mental processing using a computer mouse-tracking method , 2010, Behavior research methods.

[21]  Kerri L. Johnson,et al.  More Than Meets the Eye: Split-Second Social Perception , 2016, Trends in Cognitive Sciences.

[22]  J. Freeman,et al.  Advanced mouse-tracking analytic techniques for enhancing psychological science , 2015 .

[23]  Joseph G. Johnson,et al.  A Handbook of Process Tracing Methods , 2019 .

[24]  Heather J. Ferguson,et al.  Why We Simulate Negated Information: A Dynamic Pragmatic Account , 2010, Quarterly journal of experimental psychology.

[25]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[26]  Gregory J. Koop An assessment of the temporal dynamics of moral decisions , 2013, Judgment and Decision Making.

[27]  Michael J. Spivey,et al.  Continuous Dynamics in Real-Time Cognition , 2006 .

[28]  Roger Ratcliff,et al.  The Diffusion Decision Model: Theory and Data for Two-Choice Decision Tasks , 2008, Neural Computation.

[29]  Joseph G. Johnson,et al.  The response dynamics of preferential choice , 2013, Cognitive Psychology.

[30]  Jonathan B. Freeman,et al.  Doing Psychological Science by Hand , 2018, Current directions in psychological science.

[31]  Ye Tian,et al.  Dynamic Pragmatic View of Negation Processing , 2016 .

[32]  Michael J. Spivey,et al.  Action Dynamics Reveal Parallel Competition in Decision Making , 2008, Psychological science.

[33]  Jonathan B Freeman,et al.  A Perceptual Pathway to Bias , 2016, Psychological science.

[34]  Barbara Kaup,et al.  Event-related Potential Correlates of Negation in a Sentence-Picture Verification Paradigm , 2008, J. Cogn. Neurosci..

[35]  M. Spivey,et al.  Do curved reaching movements emerge from competing perceptions? A reply to van der Wel et al. (2009). , 2010, Journal of experimental psychology. Human perception and performance.

[36]  Thomas A. Farmer,et al.  Hand in Motion Reveals Mind in Motion , 2011, Front. Psychology.

[37]  Michael J. Spivey,et al.  Continuous attraction toward phonological competitors. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[38]  Masatoshi Koizumi,et al.  Tracking Down Disjunction , 2015, JSAI-isAI Workshops.

[39]  Rick Dale,et al.  The Self-Organization of Explicit Attitudes , 2009, Psychological science.

[40]  K. Nakayama,et al.  Role of focal attention on latencies and trajectories of visually guided manual pointing. , 2006, Journal of vision.

[41]  Felix Henninger,et al.  Mousetrap: An integrated, open-source mouse-tracking package , 2017, Behavior Research Methods.