Predicting Primary Gaze Behavior Using Social Saliency Fields

We present a method to predict primary gaze behavior in a social scene. Inspired by the study of electric fields, we posit "social charges"-latent quantities that drive the primary gaze behavior of members of a social group. These charges induce a gradient field that defines the relationship between the social charges and the primary gaze direction of members in the scene. This field model is used to predict primary gaze behavior at any location or time in the scene. We present an algorithm to estimate the time-varying behavior of these charges from the primary gaze behavior of measured observers in the scene. We validate the model by evaluating its predictive precision via cross-validation in a variety of social scenes.

[1]  Alessio Del Bue,et al.  Abnormal Crowd Behavior Detection by Social Force Optimization , 2011, HBU.

[2]  Andrew T. Duchowski,et al.  EUROGRAPHICS 2001 / Jonathan C. Roberts Short Presentations Gaze-Contingent Level Of Detail Rendering , 2022 .

[3]  C. Granger.  Investigating Causal Relations by Econometric Models and Cross-Spectral Methods , 1969 .

[4]  M. Posner,et al.  Orienting of Attention* , 1980, The Quarterly journal of experimental psychology.

[5]  N. Emery,et al.  The eyes have it: the neuroethology, function and evolution of social gaze , 2000, Neuroscience & Biobehavioral Reviews.

[6]  Mubarak Shah,et al.  Abnormal crowd behavior detection using social force model , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Anders Johansson,et al.  of a Microscopic Pedestrian Model by Evolutionary Adjustment to Video Tracking Data , 2008, 0810.4587.

[8]  Fernando De la Torre,et al.  Supervised Descent Method and Its Applications to Face Alignment , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  A. Kendon.  Conducting Interaction: Patterns of Behavior in Focused Encounters , 1990 .

[10]  Alex Pentland,et al.  To Signal Is Human , 2010 .

[11]  Helbing,et al.  Social force model for pedestrian dynamics. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[12]  Shuicheng Yan,et al.  Pair-activity classification by bi-trajectories analysis , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  R. Jampel,et al.  The primary position of the eyes, the resetting saccade, and the transverse visual head plane. Head movements around the cervical joints. , 1992, Investigative ophthalmology & visual science.

[14]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[15]  Yaser Sheikh,et al.  3D Social Saliency from Head-mounted Cameras , 2012, NIPS.

[16]  Fei-Fei Li,et al.  Social Role Discovery in Human Events , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  A. Kingstone,et al.  Human social attention. , 2009, Progress in brain research.

[18]  Larry S. Davis,et al.  Understanding videos, constructing plots learning a visually grounded storyline model from annotated videos , 2009, CVPR.

[19]  James M. Rehg,et al.  Quasi-periodic event analysis for social game retrieval , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[20]  Oleg V. Komogortsev,et al.  Perceptual attention focus prediction for multiple viewers in case of multimedia perceptual compression with feedback delay , 2006, ETRA.

[21]  James M. Rehg,et al.  Temporal causality for the analysis of visual events , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Maja Pantic,et al.  Social signal processing: Survey of an emerging domain , 2009, Image Vis. Comput..

[23]  O. Khatib,et al.  Real-Time Obstacle Avoidance for Manipulators and Mobile Robots , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[24]  Irfan A. Essa,et al.  Motion fields to predict play evolution in dynamic sport scenes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[25]  Alex Pentland,et al.  Graphical Models for Recognizing Human Interactions , 1998, NIPS.

[26]  Jake K. Aggarwal,et al.  Spatio-temporal relationship match: Video structure comparison for recognition of complex human activities , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[27]  Adam Kendon,et al.  Spacing and Orientation in Co-present Interaction , 2009, COST 2102 Training School.

[28]  Gilbert Strang,et al.  The Discrete Cosine Transform , 1999, SIAM Rev..

[29]  James M. Rehg,et al.  Categorizing Turn-Taking Interactions , 2012, ECCV.

[30]  C. Chabris,et al.  Gorillas in Our Midst: Sustained Inattentional Blindness for Dynamic Events , 1999, Perception.

[31]  E. Hall,et al.  The Hidden Dimension , 1970 .

[32]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[33]  Yvonne Rogers,et al.  Using F-formations to analyse spatial patterns of interaction in physical environments , 2011, CSCW.

[34]  Vittorio Murino,et al.  Social interactions by visual focus of attention in a three‐dimensional environment , 2013, Expert Syst. J. Knowl. Eng..

[35]  Michael Wimmer,et al.  An empirical pipeline to derive gaze prediction heuristics for 3D action games , 2010, TAP.

[36]  R. Adolphs,et al.  The social brain: neural basis of social knowledge. , 2009, Annual review of psychology.

[37]  Luc Van Gool,et al.  You'll never walk alone: Modeling social behavior for multi-target tracking , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[38]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[39]  C. Granger.  Investigating causal relations by econometric models and cross-spectral methods , 1969 .

[40]  Andrew Zisserman,et al.  Detecting People Looking at Each Other in Videos , 2014, International Journal of Computer Vision.

[41]  Yi Yang,et al.  Recognizing proxemics in personal photos , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  James M. Rehg,et al.  Social interactions: A first-person perspective , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[43]  Dinesh Manocha,et al.  Multi-robot coordination using generalized social potential fields , 2009, 2009 IEEE International Conference on Robotics and Automation.

[44]  Alessio Del Bue,et al.  Social interaction discovery by statistical analysis of F-formations , 2011, BMVC.

[45]  Silvio Savarese,et al.  Learning context for collective activity recognition , 2011, CVPR 2011.

[46]  A. Kingstone,et al.  The eyes have it! Reflexive orienting is triggered by nonpredictive gaze , 1998 .

[47]  Hongyan Wang,et al.  Social potential fields: A distributed behavioral control for autonomous robots , 1995, Robotics Auton. Syst..

[48]  Laurent Itti,et al.  Applying computational tools to predict gaze direction in interactive visual environments , 2008, TAP.

[49]  Vittorio Murino,et al.  Towards Computational Proxemics: Inferring Social Relations from Interpersonal Distances , 2011, 2011 IEEE Third Int'l Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third Int'l Conference on Social Computing.

[50]  J. Henderson.  Human gaze control during real-world scene perception , 2003, Trends in Cognitive Sciences.

[51]  Larry D. Hostetler,et al.  The estimation of the gradient of a density function, with applications in pattern recognition , 1975, IEEE Trans. Inf. Theory.

[52]  Pascal Fua,et al.  Take your eyes off the ball: Improving ball-tracking by focusing on team play , 2014, Comput. Vis. Image Underst..

[53]  Marat Sophie,et al.  Gaze Prediction Improvement by Adding a Face Feature to a Saliency Model , 2009 .

[54]  F. Shic,et al.  Decreased Spontaneous Attention to Social Scenes in 6-Month-Old Infants Later Diagnosed with Autism Spectrum Disorders , 2013, Biological Psychiatry.

[55]  Dirk Helbing,et al.  Specification of the Social Force Pedestrian Model by Evolutionary Adjustment to Video Tracking Data , 2007, Adv. Complex Syst..