Individual differences in visual salience vary along semantic dimensions

What determines where we look? Theories of attentional guidance hold that image features and task demands govern fixation behaviour, while differences between observers are ‘noise’. Here, we investigated the fixations of > 100 human adults freely viewing a large set of complex scenes. We found systematic individual differences in fixation frequencies along six semantic stimulus dimensions. These differences were large (> twofold) and highly stable across images and time. Surprisingly, they also held for first fixations directed towards each image, commonly interpreted as ‘bottom-up’ visual salience. Their perceptual relevance was documented by a correlation between individual face salience and recognition skills. The dimensions of individual salience and their covariance pattern replicated across samples from three different countries, suggesting they reflect fundamental biological mechanisms of attention. Our findings show stable individual salience differences along semantic dimensions, with meaningful perceptual implications. Salience reflects features of the observer as well as the image.

[1]  Laurent Itti,et al.  Superior colliculus neurons encode a visual saliency map during free viewing of natural dynamic video , 2017, Nature Communications.

[2]  Radoslaw Martin Cichy,et al.  Typical visual-field locations enhance processing in object-selective channels of human occipital cortex. , 2018, Journal of neurophysiology.

[3]  Nathan T. Carter,et al.  A test of the International Personality Item Pool representation of the Revised NEO Personality Inventory and development of a 120-item IPIP-based measure of the five-factor model. , 2014, Psychological assessment.

[4]  R. Hoyle,et al.  Reliability and validity of a brief measure of sensation seeking , 2002 .

[5]  James J. Clark,et al.  Personalization of Saliency Estimation , 2017, ArXiv.

[6]  A. Klin,et al.  Attention to Eyes is Present But in Decline in 2–6 Month-Olds Later Diagnosed with Autism , 2013, Nature.

[7]  Ning Qian,et al.  A Learning-Style Theory for Understanding Autistic Behaviors , 2011, Front. Hum. Neurosci..

[8]  C. Koch,et al.  Task-demands can immediately reverse the effects of sensory-driven saliency in complex visual stimuli. , 2008, Journal of vision.

[9]  Leon A. Gatys,et al.  Understanding Low- and High-Level Contributions to Fixation Prediction , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[10]  Bruce D. McCandliss,et al.  The visual word form area: expertise for reading in the fusiform gyrus , 2003, Trends in Cognitive Sciences.

[11]  R. Gregory The Most Expensive Painting in the World , 2007, Perception.

[12]  Derrick J. Parkhurst,et al.  Modeling the role of salience in the allocation of overt visual attention , 2002, Vision Research.

[13]  Rémy Allard,et al.  Common first- and second-order motion processing at high temporal frequencies , 2010 .

[14]  Thomas Armstrong,et al.  Eye tracking of attention in the affective disorders: a meta-analytic review and synthesis. , 2012, Clinical psychology review.

[15]  Pierre-Yves Oudeyer,et al.  Eye movements reveal epistemic curiosity in human observers , 2015, Vision Research.

[16]  Corinne C Stauffer,et al.  Processing Visual Temporal Information and Its Relationship to Psychometric Intelligence Converging Evidence for the Temporal Resolution Power Hypothesis of Intelligence , 2011 .

[17]  G R Loftus,et al.  The functional visual field during picture viewing. , 1980, Journal of experimental psychology. Human learning and memory.

[18]  Carrick C. Williams,et al.  Eye movements and picture processing during recognition , 2003, Perception & psychophysics.

[19]  Brian J. Scholl,et al.  Attentive tracking of objects vs. substances , 2010 .

[20]  Mary H. MacLean,et al.  The “working” component of working memory predicts AB magnitude , 2010 .

[21]  Agostino Gibaldi,et al.  Evaluation of the Tobii EyeX Eye tracking controller and Matlab toolkit for research , 2016, Behavior Research Methods.

[22]  K. Gegenfurtner The Interaction Between Vision and Eye Movements † , 2016, Perception.

[23]  K. Nakayama,et al.  The Cambridge Face Memory Test: Results for neurologically intact individuals and an investigation of its validity using inverted face stimuli and prosopagnosic participants , 2006, Neuropsychologia.

[24]  Ruth E. Hogg,et al.  Individual differences in human eye movements: An oculomotor signature? , 2017, Vision Research.

[25]  Tal Makovski,et al.  The visual attractor illusion. , 2010, Journal of vision.

[26]  Doris Y. Tsao,et al.  Comparing face patch systems in macaques and humans , 2008, Proceedings of the National Academy of Sciences.

[27]  Nicola C. Anderson,et al.  The influence of personality on social attention , 2014 .

[28]  P. Costa,et al.  Four ways five factors are basic , 1992 .

[29]  K. Grill-Spector,et al.  Neural representations of faces and limbs neighbor in human high-level visual cortex: evidence for a new organization principle , 2011, Psychological Research.

[30]  S. Holm A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[31]  Nicola C. Anderson,et al.  Curious eyes: Individual differences in personality predict eye movement behavior in scene-viewing , 2012, Cognition.

[32]  E. Zohary,et al.  Topographic Representation of the Human Body in the Occipitotemporal Cortex , 2010, Neuron.

[33]  Philip C. Ko,et al.  Eye movements in Alzheimer's disease. , 2015, Journal of Alzheimer's disease : JAD.

[34]  Marcus Nyström,et al.  Semantic override of low-level features in image viewing - both initially and overall , 2008 .

[35]  D. Schwarzkopf,et al.  Feature-location effects in the Thatcher illusion. , 2018, Journal of vision.

[36]  Shuo Wang,et al.  Predicting human gaze beyond pixels. , 2014, Journal of vision.

[37]  N. Kanwisher,et al.  The fusiform face area: a cortical region specialized for the perception of faces , 2006, Philosophical Transactions of the Royal Society B: Biological Sciences.

[38]  Zhenzhong Chen,et al.  Personalized Visual Saliency: Individuality Affects Image Perception , 2018, IEEE Access.

[39]  Denis G. Pelli,et al.  ECVP '07 Abstracts , 2007, Perception.

[40]  A. Klin,et al.  Infant viewing of social scenes is under genetic control and atypical in autism , 2017, Nature.

[41]  K. Bertsch,et al.  Variance in saccadic eye movements reflects stable traits. , 2016, Psychophysiology.

[42]  Stefan J. Troche,et al.  Attentional blink and impulsiveness: evidence for higher functional impulsivity in non-blinkers compared to blinkers , 2013, Cognitive Processing.

[43]  M. Gamer,et al.  Orienting towards social features in naturalistic scenes is reflexive , 2017, PloS one.

[44]  C. Curcio,et al.  Topography of ganglion cells in human retina , 1990, The Journal of comparative neurology.

[45]  A. Aron,et al.  Sensory-processing sensitivity and its relation to introversion and emotionality. , 1997, Journal of personality and social psychology.

[46]  W. Einhäuser,et al.  Overt attention in natural scenes: Objects dominate features , 2015, Vision Research.

[47]  B. Fischer,et al.  Visual field representations and locations of visual areas V1/2/3 in human visual cortex. , 2003, Journal of vision.

[48]  Sébastien M. Crouzet,et al.  Fast saccades toward faces: face detection in just 100 ms. , 2010, Journal of vision.

[49]  Nancy Kanwisher,et al.  Delay of selective attention during the attentional blink , 2010 .

[50]  Ioannis Rigas,et al.  Current research in eye movement biometrics: An analysis based on BioEye 2015 competition , 2017, Image Vis. Comput..

[51]  Andreas Bulling,et al.  Recognition of curiosity using eye movement analysis , 2015, UbiComp/ISWC Adjunct.

[52]  Taylor R. Hayes,et al.  Scan patterns during scene viewing predict individual differences in clinical traits in a normative sample , 2018, PloS one.

[53]  J. Wilmer How to use individual differences to isolate functional organization, biology, and utility of visual functions; with illustrative proposals for stereopsis. , 2008, Spatial vision.

[54]  Daniel P. Kennedy,et al.  Viewing Complex, Dynamic Scenes “Through the Eyes” of Another Person: The Gaze-Replay Paradigm , 2015, PloS one.

[55]  John M Henderson,et al.  Stable individual differences across images in human saccadic eye movements. , 2008, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[56]  L. Itti,et al.  Defending Yarbus: eye movements reveal observers' task. , 2014, Journal of vision.

[57]  Geraint Rees,et al.  Perception and Processing of Faces in the Human Brain Is Tuned to Typical Feature Locations , 2016, The Journal of Neuroscience.

[58]  D. Ballard,et al.  Eye guidance in natural vision: reinterpreting salience. , 2011, Journal of vision.

[59]  Keith Rayner,et al.  The mask-onset delay paradigm and the availability of central and peripheral visual information during scene viewing. , 2012, Journal of vision.

[60]  T. Foulsham,et al.  What can saliency models predict about eye movements? Spatial and sequential aspects of fixations during encoding and recognition. , 2008, Journal of vision.

[61]  C. Koch,et al.  Evidence for two distinct mechanisms directing gaze in natural scenes. , 2012, Journal of vision.

[62]  Rufin Vogels,et al.  A behavioral face preference deficit in a monkey with an incomplete face patch system , 2019, NeuroImage.

[63]  G. T. Buswell How People Look At Pictures: A Study Of The Psychology Of Perception In Art , 2012 .

[64]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[65]  K. Grill-Spector,et al.  The functional architecture of the ventral temporal cortex and its role in categorization , 2014, Nature Reviews Neuroscience.

[66]  J. Haynes,et al.  Decoding Successive Computational Stages of Saliency Processing , 2011, Current Biology.

[67]  Daniel P. Kennedy,et al.  Atypical Visual Saliency in Autism Spectrum Disorder Quantified through Model-Based Eye Tracking , 2015, Neuron.

[68]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[69]  N. Kanwisher,et al.  Activation in Human MT/MST by Static Images with Implied Motion , 2000, Journal of Cognitive Neuroscience.

[70]  P. Perona,et al.  Objects predict fixations better than early saliency. , 2008, Journal of vision.

[71]  Matthias Bethge,et al.  Information-theoretic model comparison unifies saliency metrics , 2015, Proceedings of the National Academy of Sciences.

[72]  Vanessa Troiani,et al.  Distinct and overlapping fusiform activation to faces and food , 2018, NeuroImage.

[73]  Shenghua Gao,et al.  Personalized Saliency and Its Prediction , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[74]  Margaret S. Livingstone,et al.  Seeing faces is necessary for face-patch formation , 2017, Nature Neuroscience.

[75]  Jennifer C. Sarrett,et al.  Commentary: Attention to Eyes Is Present but in Decline in 2–6-Month-Old Infants Later Diagnosed with Autism , 2015, Front. Public Health.

[76]  Daniel P. Kennedy,et al.  Genetic Influence on Eye Movements to Complex Scenes at Short Timescales , 2017, Current Biology.

[77]  Thomas Martinetz,et al.  Variability of eye movements when viewing dynamic natural scenes. , 2010, Journal of vision.

[78]  Julie E. Boland,et al.  Cultural variation in eye movements during scene perception. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[79]  Taylor R. Hayes,et al.  Meaning guides attention in real-world scene images: Evidence from eye movements and meaning maps , 2017, bioRxiv.

[80]  S. Shevell,et al.  Multistable binocular feature-integrated percepts are frozen by intermittent presentation. , 2011, Journal of vision.

[81]  Ulrich Ettinger,et al.  Reliability of smooth pursuit, fixation, and saccadic eye movements. , 2003, Psychophysiology.

[82]  Steven G. Luke,et al.  Stable individual differences in saccadic eye movements during reading, pseudoreading, scene viewing, and scene search. , 2014, Journal of experimental psychology. Human perception and performance.

[83]  N. Kriegeskorte,et al.  The brain of the beholder: honouring individual representational idiosyncrasies , 2015, bioRxiv.

[84]  David H. Peterzell,et al.  Discovering Sensory Processes Using Individual Differences: A Review and Factor Analytic Manifesto , 2016, HVEI.

[85]  S. Dehaene,et al.  How Learning to Read Changes the Cortical Networks for Vision and Language , 2010, Science.

[86]  A. L. Yarbus Eye Movements During Perception of Complex Objects , 1967 .

[87]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[88]  R. Rosenholtz Capabilities and Limitations of Peripheral Vision. , 2016, Annual review of vision science.

[89]  D. Coppola,et al.  Idiosyncratic characteristics of saccadic eye movements when viewing different visual environments , 1999, Vision Research.

[90]  Hans Strasburger,et al.  James Jurin (1684-1750): a pioneer of crowding research? , 2015, Journal of vision.

[91]  Nicola C. Anderson,et al.  It depends on when you look at it: Salience influences eye movements in natural scene viewing and search early in time. , 2015, Journal of vision.

[92]  Taylor R. Hayes,et al.  Meaning-based guidance of attention in scenes as revealed by meaning maps , 2017, Nature Human Behaviour.

[93]  Taylor R. Hayes,et al.  Scan patterns during real-world scene viewing predict individual differences in cognitive capacity. , 2017, Journal of vision.

[94]  F. Volkmar,et al.  Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. , 2002, Archives of general psychiatry.