What do you want to do next: a novel approach for intent prediction in gaze-based interaction

Interaction intent prediction and the Midas touch have been a longstanding challenge for eye-tracking researchers and users of gaze-based interaction. Inspired by machine learning approaches in biometric person authentication, we developed and tested an offline framework for task-independent prediction of interaction intents. We describe the principles of the method, the features extracted, normalization methods, and evaluation metrics. We systematically evaluated the proposed approach on an example dataset of gaze-augmented problem-solving sessions. We present results of three normalization methods, different feature sets and fusion of multiple feature types. Our results show that accuracy of up to 76% can be achieved with Area Under Curve around 80%. We discuss the possibility of applying the results for an online system capable of interaction intent prediction.

[1]  Samuel Kaski,et al.  Inferring Relevance from Eye Movements: Feature Extraction , 2005 .

[2]  Brian P. Bailey,et al.  Understanding changes in mental workload during execution of goal-directed tasks and its application for interruption management , 2008, TCHI.

[3]  B. P. Bogert,et al.  The quefrency analysis of time series for echoes : cepstrum, pseudo-autocovariance, cross-cepstrum and saphe cracking , 1963 .

[4]  Eric Jones,et al.  SciPy: Open Source Scientific Tools for Python , 2001 .

[5]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[6]  A. W. M. van den Enden,et al.  Discrete Time Signal Processing , 1989 .

[7]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[8]  D. Kahneman Attention and Effort , 1973 .

[9]  J. Beatty Task-evoked pupillary responses, processing load, and the structure of processing resources. , 1982 .

[10]  Keith S. Karn,et al.  Commentary on Section 4. Eye tracking in human-computer interaction and usability research: Ready to deliver the promises. , 2003 .

[11]  Jean Lorenceau,et al.  Pupil dynamics during bistable motion perception. , 2009, Journal of vision.

[12]  Tomi Kinnunen,et al.  Towards task-independent person authentication using eye movement signals , 2010, ETRA.

[13]  Brian P. Bailey,et al.  Categories & Subject Descriptors: H.5.2 [Information , 2022 .

[14]  Shahram Eivazi,et al.  Predicting Problem-Solving Behavior and Performance Levels from Visual Attention Data , 2011 .

[15]  Pawel Kasprowski,et al.  Eye Movements in Biometrics , 2004, ECCV Workshop BioAW.

[16]  John Shawe-Taylor,et al.  Can eyes reveal interest? Implicit queries from gaze patterns , 2009, User Modeling and User-Adapted Interaction.

[17]  Howell O. Istance,et al.  Snap clutch, a moded approach to solving the Midas touch problem , 2008, ETRA.

[18]  Markku Tukiainen,et al.  Gaze interaction enhances problem solving: Effects of dwell-time based, gaze-augmented, and mouse interaction on problem-solving strategies and user experience , 2009 .

[19]  J. Beatty,et al.  Pupillary dilations in movement preparation and execution. , 1985, Psychophysiology.

[20]  Ingo Mierswa,et al.  YALE: rapid prototyping for complex data mining tasks , 2006, KDD '06.

[21]  Päivi Majaranta,et al.  Twenty years of eye typing: systems and design issues , 2002, ETRA.

[22]  Ilpo Kojo,et al.  Using hidden Markov model to uncover processing states from eye movements in information search tasks , 2008, Cognitive Systems Research.

[23]  Alan V. Oppenheim,et al.  Discrete-time signal processing (2nd ed.) , 1999 .

[24]  Alex Waibel,et al.  Readings in speech recognition , 1990 .

[25]  Samuel Kaski,et al.  Inferring object relevance from gaze in dynamic scenes , 2010, ETRA.

[26]  Tomi Kinnunen,et al.  Eye-Movements as a Biometric , 2005, SCIA.

[27]  Jeff Klingner,et al.  Fixation-aligned pupillary response averaging , 2010, ETRA.

[28]  James P. Egan,et al.  Signal detection theory and ROC analysis , 1975 .

[29]  Stephen J. Payne,et al.  Skim reading by satisficing: evidence from eye tracking , 2011, CHI.

[30]  Ron Kohavi,et al.  The Case against Accuracy Estimation for Comparing Induction Algorithms , 1998, ICML.

[31]  J. Beatty,et al.  The pupillary system. , 2000 .