Strokes of insight: User intent detection and kinematic compression of mouse cursor trails

We use the Kinematic Theory to detect user intent in mouse cursor trails.As a practical application, we propose a novel (kinematic) compression technique.This work enables a deeper understanding of mouse cursor movements production. Web users often have a specific goal in mind comprising various stages that are reflected, as executed, by their mouse cursor movements. Therefore, is it possible to detect automatically which parts of those movements bear any intent and discard the parts that have no intent? Can we estimate the intent degree of the non-discarded parts? To achieve this goal, we tap into the Kinematic Theory and its associated Sigma-Lognormal model (ΣźM). According to this theory, the production of a mouse cursor movement requires beforehand the instantiation of an action plan. The ΣźM models such an action plan as a sequence of strokes' velocity profiles, one stroke at a time, providing thus a reconstruction of the original mouse cursor movement. When a user intent is clear, the pointing movement is faster and the cursor movement is reconstructed almost perfectly, while the reverse is observed when the user intent is unclear.We analyzed more than 10,000 browsing sessions comprising about 5 million of data points, and compared different segmentation techniques to detect discrete cursor chunks that were then reconstructed with the ΣźM. Our main contribution is thus a novel methodology to automatically tell chunks with and without intention apart. We also contribute with kinematic compression, a novel application to compress mouse cursor data while preserving most of the original information. Ultimately, this work enables a deeper understanding of mouse cursor movements production, providing an informed means to gain additional insight about users' browsing behavior.

[1]  Réjean Plamondon,et al.  A kinematic theory of rapid human movements , 1995, Biological Cybernetics.

[2]  R Plamondon,et al.  Studying the variability of handwriting patterns using the Kinematic Theory. , 2009, Human movement science.

[3]  J. J. Denier,et al.  The guiding of human writing movements , 1965, Kybernetik.

[4]  Luis A. Leiva,et al.  Assessing users' interactions for clustering web documents: a pragmatic approach , 2010, HT '10.

[5]  M. Brass,et al.  Unconscious determinants of free decisions in the human brain , 2008, Nature Neuroscience.

[6]  Réjean Plamondon,et al.  Training of On-Line Handwriting Text Recognizers with Synthetic Text Generated Using the Kinematic Theory of Rapid Human Movements , 2014, 2014 14th International Conference on Frontiers in Handwriting Recognition.

[7]  Ryen W. White,et al.  Leaving so soon?: understanding and predicting web search abandonment rationales , 2012, CIKM.

[8]  M. Braga,et al.  Exploratory Data Analysis , 2018, Encyclopedia of Social Network Analysis and Mining. 2nd Ed..

[9]  Mark Claypool,et al.  Implicit interest indicators , 2001, IUI '01.

[10]  Réjean Plamondon,et al.  A kinematic theory of rapid human movements , 1995, Biological Cybernetics.

[11]  Joemon M. Jose,et al.  User engagement in online News: Under the scope of sentiment, interest, affect, and gaze , 2014, J. Assoc. Inf. Sci. Technol..

[12]  A. Savitzky,et al.  Smoothing and Differentiation of Data by Simplified Least Squares Procedures. , 1964 .

[13]  Yoshinori Hijikata,et al.  Implicit user profiling for on demand relevance feedback , 2004, IUI '04.

[14]  Eugene Agichtein,et al.  Exploring mouse movements for inferring query intent , 2008, SIGIR '08.

[15]  T. Flash,et al.  The coordination of arm movements: an experimentally confirmed mathematical model , 1985, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[16]  T. Flash,et al.  Minimum-jerk, two-thirds power law, and isochrony: converging approaches to movement planning. , 1995, Journal of experimental psychology. Human perception and performance.

[17]  R. Plamondon,et al.  A multi-level representation paradigm for handwriting stroke generation. , 2006, Human movement science.

[18]  Réjean Plamondon,et al.  Modelization of Handwriting: A System Approach , 1986 .

[19]  George Valkanas,et al.  Understanding Within-Content Engagement through Pattern Analysis of Mouse Gestures , 2014, CIKM.

[20]  J. Hollerbach An oscillation theory of handwriting , 2004, Biological Cybernetics.

[21]  Edward Lank,et al.  Endpoint prediction using motion kinematics , 2007, CHI.

[22]  Réjean Plamondon,et al.  Development of a Sigma-Lognormal representation for on-line signatures , 2009, Pattern Recognit..

[23]  Stephen Grossberg,et al.  The Vite Model: A Neural Command Circuit for Generating Arm and Articulator Trajectories, , 1988 .

[24]  B. Libet,et al.  Time of conscious intention to act in relation to onset of cerebral activity (readiness-potential). The unconscious initiation of a freely voluntary act. , 1983, Brain : a journal of neurology.

[25]  P D Neilson,et al.  The problem of redundancy in movement control: The adaptive model theory approach , 1993, Psychological research.

[26]  P. Viviani,et al.  The law relating the kinematic and figural aspects of drawing movements. , 1983, Acta psychologica.

[27]  Yiqun Liu,et al.  Different Users, Different Opinions: Predicting Search Satisfaction with Mouse Movement Information , 2015, SIGIR.

[28]  Bracha Shapira,et al.  Study of the usefulness of known and new implicit indicators and their optimal combination for accurate inference of users interests , 2006, SAC.

[29]  Albrecht Schmidt,et al.  Knowing the User's Every Move – User Activity Tracking for Website Usability Evaluation and Implicit Interaction , 2006 .

[30]  Christian O'Reilly,et al.  The lognormal handwriter: learning, performing, and declining , 2013, Front. Psychol..

[31]  Christopher P. Holland,et al.  The effect of prior knowledge and decision-making style on the online purchase decision-making process: A typology of consumer shopping behaviour , 2015, Decis. Support Syst..

[32]  Luis A. Leiva,et al.  Gestures à Go Go , 2015, ACM Trans. Intell. Syst. Technol..

[33]  Jude W. Shavlik,et al.  Learning users' interests by unobtrusively observing their normal behavior , 2000, IUI '00.

[34]  Abigail Sellen,et al.  How knowledge workers use the web , 2002, CHI.

[35]  Martin Gaedke,et al.  TellMyRelevance!: predicting the relevance of web search results from cursor interactions , 2013, CIKM.

[36]  Patrick Haggard,et al.  Decision Time for Free Will , 2011, Neuron.

[37]  Luis A. Leiva,et al.  Building a better mousetrap: Compressing mouse cursor activity for web analytics , 2015, Inf. Process. Manag..

[38]  Réjean Plamondon,et al.  Improving sigma-lognormal parameter extraction , 2015, 2015 13th International Conference on Document Analysis and Recognition (ICDAR).

[39]  Ryen W. White,et al.  No clicks, no problem: using cursor movements to understand and improve search , 2011, CHI.

[40]  Luis A. Leiva,et al.  Predicting User Engagement with Direct Displays Using Mouse Cursor Information , 2016, SIGIR.

[41]  Christian O'Reilly,et al.  Recent developments in the study of rapid human movements with the kinematic theory: Applications to handwriting and signature synthesis , 2014, Pattern Recognit. Lett..

[42]  Bonnie MacKay,et al.  An examination of multisession web tasks , 2012, J. Assoc. Inf. Sci. Technol..

[43]  Eugene Agichtein,et al.  Beyond dwell time: estimating document relevance from cursor movements and other post-click searcher behavior , 2012, WWW.

[44]  S. Glover,et al.  Separate visual representations in the planning and control of action , 2004, Behavioral and Brain Sciences.

[45]  Leclerc,et al.  4 - Des gaussiennes pour la modélisation des signatures et la segmentation de tracés manuscrits , 1992 .

[46]  Eugene Agichtein,et al.  Ready to buy or just browsing?: detecting web searcher goals from interaction data , 2010, SIGIR.

[47]  Luis A. Leiva,et al.  Web browsing behavior analysis and interactive hypervideo , 2013, TWEB.

[48]  Eamonn J. Keogh,et al.  Segmenting Time Series: A Survey and Novel Approach , 2002 .

[49]  Eugene Agichtein,et al.  Discovering common motifs in cursor movement data for improving web search , 2014, WSDM.

[50]  Réjean Plamondon,et al.  Synthetic on-line signature generation. Part I: Methodology and algorithms , 2012, Pattern Recognit..

[51]  Pietro Morasso,et al.  How a discontinuous mechanism can produce continuous patterns in trajectory formation and handwriting , 1983 .

[52]  F. J. Maarse The study of handwriting movement: peripheral models and signal processing techniques , 1987 .

[53]  Ryen W. White,et al.  Improving searcher models using mouse cursor activity , 2012, SIGIR '12.

[54]  David E. Meyer,et al.  Speed—Accuracy Tradeoffs in Aimed Movements: Toward a Theory of Rapid Voluntary Action , 2018, Attention and Performance XIII.

[55]  Ernesto Arroyo,et al.  Usability tool for analysis of web designs using mouse tracks , 2006, CHI Extended Abstracts.

[56]  A. Alimi Beta neuro-fuzzy systems , 2003 .

[57]  Réjean Plamondon,et al.  Can computer mice be used as low-cost devices for the acquisition of planar human movement velocity signals? , 2011, Behavior research methods.

[58]  Réjean Plamondon,et al.  Modelling velocity profiles of rapid movements: a comparative study , 1993, Biological Cybernetics.