Detecting expert’s eye using a multiple-kernel Relevance Vector Machine

Decoding mental states from the pattern of neural activity or overt behavior is an intensely pursued goal. Here we applied machine learning to detect expertise from the oculomotor behavior of novice and expert billiard players during free viewing of a filmed billiard match with no specific task, and in a dynamic trajectory prediction task involving ad-hoc, occluded billiard shots. We have adopted a ground framework for feature space fusion and a Bayesian sparse classifier, namely, a Relevance Vector Machine. By testing different combinations of simple oculomotor features (gaze shifts amplitude and direction, and fixation duration), we could classify on an individual basis which group - novice or expert - the observers belonged to with an accuracy of 82% and 87%, respectively for the match and the shots. These results provide evidence that, at least in the particular domain of billiard sport, a signature of expertise is hidden in very basic aspects of oculomotor behavior, and that expertise can be detected at the individual level both with ad-hoc testing conditions and under naturalistic conditions - and suitable data mining. Our procedure paves the way for the development of a test for the “expert’s eye”, and promotes the use of eye movements as an additional signal source in Brain-Computer-Interface (BCI) systems.

[1]  Roman Bednarik,et al.  What do you want to do next: a novel approach for intent prediction in gaze-based interaction , 2012, ETRA.

[2]  Reza Fazel-Rezai,et al.  A Review of Hybrid Brain-Computer Interface Systems , 2013, Adv. Hum. Comput. Interact..

[3]  Paolo Napoletano,et al.  Bayesian Integration of Face and Low-Level Cues for Foveated Video Coding , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[4]  L. Stark,et al.  Spontaneous Eye Movements During Visual Imagery Reflect the Content of the Visual Scene , 1997, Journal of Cognitive Neuroscience.

[5]  E. Barth,et al.  Efficient visual coding and the predictability of eye movements on natural movies. , 2009, Spatial vision.

[6]  Andreas Bulling,et al.  Wearable eye tracking for mental health monitoring , 2012, Comput. Commun..

[7]  Claudio de’Sperati,et al.  The Inner Working of Dynamic Visuo-Spatial Imagery as Revealed by Spontaneous Eye Movements , 2003 .

[8]  L. Itti,et al.  High-throughput classification of clinical populations from natural viewing eye movements , 2012, Journal of Neurology.

[9]  T. Donovan,et al.  The radiology task: Bayesian theory and perception. , 2007, The British journal of radiology.

[10]  Carlo Robino,et al.  Parsing visual stimuli into temporal units through eye movements , 2012, ETRA.

[11]  K. Fernow New York , 1896, American Potato Journal.

[12]  Svein Magnussen,et al.  Expertise in Pictorial Perception: Eye-Movement Patterns and Visual Memory in Artists and Laymen , 2007, Perception.

[13]  Gary Feng,et al.  Eye movements as time-series random variables: A stochastic model of eye movement control in reading , 2006, Cognitive Systems Research.

[14]  Tomi Kinnunen,et al.  Towards task-independent person authentication using eye movement signals , 2010, ETRA.

[15]  Colin Campbell,et al.  Inferring Sparse Kernel Combinations and Relevance Vectors: An Application to Subcellular Localization of Proteins , 2008, 2008 Seventh International Conference on Machine Learning and Applications.

[16]  Nancy Millette,et al.  How People Look at Pictures , 1935 .

[17]  Gerhard Tröster,et al.  Eye Movement Analysis for Activity Recognition Using Electrooculography , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[19]  Theodoros Damoulas,et al.  Multiclass Relevance Vector Machines: Sparsity and Accuracy , 2010, IEEE Transactions on Neural Networks.

[20]  Jan Theeuwes,et al.  ScanMatch: A novel method for comparing fixation sequences , 2010, Behavior research methods.

[21]  Claudio de'Sperati,et al.  Smooth pursuit-like eye movements during mental extrapolation of motion: the facilitatory effect of drowsiness. , 2005, Brain research. Cognitive brain research.

[22]  G Underwood,et al.  Studying expertise in music reading: Use of a pattern-matching paradigm , 1997, Perception & psychophysics.

[23]  C. Patterson,et al.  Detecting Cognitive Impairment in the Elderly , 1990 .

[24]  E. Reingold,et al.  Eye movements and visual expertise in chess and medicine , 2011 .

[25]  L W Stark,et al.  String editing analysis of human visual search. , 1995, Optometry and vision science : official publication of the American Academy of Optometry.

[26]  A. Billard,et al.  Investigating Gaze of Children with ASD in Naturalistic Settings , 2012, PloS one.

[27]  Lawrence W. Stark,et al.  Visual perception and sequences of eye movement fixations: a stochastic modeling approach , 1992, IEEE Trans. Syst. Man Cybern..

[28]  Claudio M. Privitera,et al.  Algorithms for Defining Visual Regions-of-Interest: Comparison with Eye Fixations , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  S. Crespi,et al.  Spotting expertise in the eyes: billiards knowledge as revealed by gaze shifts in a dynamic visual prediction task. , 2012, Journal of vision.

[30]  Guang-Zhong Yang,et al.  Analysis of visual search patterns with EMD metric in normalized anatomical space , 2006, IEEE Transactions on Medical Imaging.

[31]  Richard Dewhurst,et al.  Eye movements during scene recollection have a functional role, but they are not reinstatements of those produced during encoding. , 2012, Journal of experimental psychology. Human perception and performance.

[32]  Pilar Orero,et al.  Algorithm for Discriminating Aggregate Gaze Points: Comparison with Salient Regions-Of-Interest , 2010, ACCV Workshops.

[33]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[34]  Theo Geisel,et al.  The ecology of gaze shifts , 2000, Neurocomputing.

[35]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevance Vector Machine , 2001 .

[36]  P. Viviani Eye movements in visual search: cognitive, perceptual and motor control aspects. , 1990, Reviews of oculomotor research.

[37]  K. Rayner Eye movements in reading and information processing: 20 years of research. , 1998, Psychological bulletin.

[38]  G. Underwood,et al.  Domain knowledge moderates the influence of visual saliency in scene recognition. , 2009, British journal of psychology.

[39]  Heiner Deubel,et al.  Time gaps in mental imagery introduced by competing saccadic tasks , 2009, Vision Research.

[40]  Benjamin W. Tatler,et al.  Systematic tendencies in scene viewing , 2008 .

[41]  H. Simon,et al.  Perception in chess , 1973 .

[42]  L. Stark,et al.  Scanpaths in Eye Movements during Pattern Perception , 1971, Science.

[43]  T. Foulsham,et al.  What can saliency models predict about eye movements? Spatial and sequential aspects of fixations during encoding and recognition. , 2008, Journal of vision.

[44]  G. Underwood Eye guidance in reading and scene perception , 1998 .

[45]  Giuseppe Boccignone,et al.  Gaze shift behavior on video as composite information foraging , 2013, Signal Process. Image Commun..

[46]  Peter König,et al.  Salient features in gaze-aligned recordings of human visual input during free exploration of natural environments. , 2008, Journal of vision.

[47]  Marcus Nyström,et al.  A vector-based, multidimensional scanpath similarity measure , 2010, ETRA.

[48]  Sébastien Miellet,et al.  iMap: a novel method for statistical fixation mapping of eye movement data , 2011, Behavior research methods.

[49]  Roel Vertegaal,et al.  The Attentive Hearing Aid: Eye Selection of Auditory Sources for Hearing Impaired Users , 2009, INTERACT.

[50]  Oleg V. Komogortsev,et al.  Biometric identification via eye movement scanpaths in reading , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[51]  Theodoros Damoulas,et al.  Pattern recognition with a Bayesian kernel combination machine , 2009, Pattern Recognit. Lett..

[52]  Shahram Eivazi,et al.  Predicting Problem-Solving Behavior and Performance Levels from Visual Attention Data , 2011 .

[53]  A. L. I︠A︡rbus Eye Movements and Vision , 1967 .

[54]  S R Ellis,et al.  Statistical Dependency in Visual Scanning , 1986, Human factors.

[55]  David J. Weiss,et al.  Performance-based assessment of expertise: How to decide if someone is an expert or not , 2002, Eur. J. Oper. Res..

[56]  Anthonius J.M. de Jong,et al.  Types and qualities of knowledge , 1993 .

[57]  L. Stark,et al.  Evidence for a global scanpath strategy in viewing abstract compared with realistic images , 1995, Neuropsychologia.

[58]  Per B Sederberg,et al.  A novel method for analyzing sequential eye movements reveals strategic influence on Raven's Advanced Progressive Matrices. , 2011, Journal of vision.

[59]  John M. Henderson,et al.  Predicting Cognitive State from Eye Movements , 2013, PloS one.

[60]  J. Vickers Perception, Cognition, and Decision Training: The Quiet Eye in Action , 2007 .

[61]  Michael E. Holmes,et al.  Clutter or content?: how on-screen enhancements affect how TV viewers scan and what they learn , 2006, ETRA.

[62]  David J. C. MacKay,et al.  Bayesian Interpolation , 1992, Neural Computation.

[63]  Ali Borji,et al.  State-of-the-Art in Visual Attention Modeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[64]  Theodoros Damoulas,et al.  Pattern Recognition , 1998, Encyclopedia of Information Systems.

[65]  H L Kundel,et al.  Nature of expertise in searching mammograms for breast masses , 1996, Medical Imaging.

[66]  A. L. Yarbus,et al.  Eye Movements and Vision , 1967, Springer US.

[67]  Giuseppe Boccignone,et al.  Feed and fly control of visual scanpaths for foveation image processing , 2012, annals of telecommunications - annales des télécommunications.

[68]  B. Tatler,et al.  The prominence of behavioural biases in eye guidance , 2009 .

[69]  R. Hari,et al.  Experiencing Art: The Influence of Expertise and Painting Abstraction Level , 2011, Front. Hum. Neurosci..

[70]  E. Krupinski,et al.  The Role of Formal Art Training on Perception and Aesthetic Judgment of Art Compositions , 2017 .

[71]  Eugene Agichtein,et al.  Detecting cognitive impairment by eye movement analysis using automatic classification algorithms , 2011, Journal of Neuroscience Methods.

[72]  Claudio de’Sperati,et al.  Saccades to mentally rotated targets , 1999, Experimental Brain Research.

[73]  Tomi Kinnunen,et al.  Eye-Movements as a Biometric , 2005, SCIA.

[74]  Giuseppe Boccignone,et al.  Ecological Sampling of Gaze Shifts , 2014, IEEE Transactions on Cybernetics.

[75]  S. Silvoni,et al.  Brain-Computer Interface in Stroke: A Review of Progress , 2011, Clinical EEG and neuroscience.

[76]  Geri Gay,et al.  Averaging scan patterns and what they can tell us , 2006, ETRA.

[77]  J. Henderson Human gaze control during real-world scene perception , 2003, Trends in Cognitive Sciences.