Coarse-to-fine information integration in human vision

&NA; Coarse‐to‐fine theories of vision propose that the coarse information carried by the low spatial frequencies (LSF) of visual input guides the integration of finer, high spatial frequency (HSF) detail. Whether and how LSF modulates HSF processing in naturalistic broad‐band stimuli is still unclear. Here we used multivariate decoding of EEG signals to separate the respective contribution of LSF and HSF to the neural response evoked by broad‐band images. Participants viewed images of human faces, monkey faces and phase‐scrambled versions that were either broad‐band or filtered to contain LSF or HSF. We trained classifiers on EEG scalp‐patterns evoked by filtered scrambled stimuli and evaluated the derived models on broad‐band scrambled and intact trials. We found reduced HSF contribution when LSF was informative towards image content, indicating that coarse information does guide the processing of fine detail, in line with coarse‐to‐fine theories. We discuss the potential cortical mechanisms underlying such coarse‐to‐fine feedback.

[1]  R Jones,et al.  Visual evoked response as a function of grating spatial frequency. , 1978, Investigative ophthalmology & visual science.

[2]  Lars Muckli,et al.  Cortical feedback signals generalise across different spatial frequencies of feedforward inputs , 2017, NeuroImage.

[3]  D. Heeger,et al.  Linear Systems Analysis of Functional Magnetic Resonance Imaging in Human V1 , 1996, The Journal of Neuroscience.

[4]  J. Robson.  Spatial and Temporal Contrast-Sensitivity Functions of the Visual System , 1966 .

[5]  S. Dehaene,et al.  Characterizing the dynamics of mental representations: the temporal generalization method , 2014, Trends in Cognitive Sciences.

[6]  D M Parker,et al.  Simple reaction times to the onset, onset, and contrast reversal of sinusoidal grating stimuli , 1980, Perception & psychophysics.

[7]  Nathalie Guyader,et al.  Coarse-to-fine Categorization of Visual Scenes in Scene-selective Cortex , 2014, Journal of Cognitive Neuroscience.

[8]  Milena Mihaylova,et al.  Peripheral and central delay in processing high spatial frequencies: reaction time and VEP latency studies , 1999, Vision Research.

[9]  J. Hegdé.  Time course of visual perception: Coarse-to-fine processing and beyond , 2008, Progress in Neurobiology.

[10]  J. Bullier.  Integrated model of visual processing , 2001, Brain Research Reviews.

[11]  Rainer Goebel,et al.  From Coarse to Fine? Spatial and Temporal Dynamics of Cortical Face Processing , 2010, Cerebral cortex.

[12]  D. Jeffreys,et al.  The influence of spatial frequency on the reaction times and evoked potentials recorded to grating pattern stimuli , 1985, Vision Research.

[13]  Peter J. Burt,et al.  Smart sensing within a pyramid vision machine , 1988, Proc. IEEE.

[14]  J R Lishman,et al.  Evidence for the View That Temporospatial Integration in Vision is Temporally Anisotropic , 1997, Perception.

[15]  I. Gauthier,et al.  Spatial scale contribution to early visual differences between face and object processing. , 2003, Brain research. Cognitive brain research.

[16]  M. Bar.  A Cortical Mechanism for Triggering Top-Down Facilitation in Visual Object Recognition , 2003, Journal of Cognitive Neuroscience.

[17]  J. Sergent.  Microgenesis of Face Perception , 1986 .

[18]  Karl J. Friston,et al.  A Parametric Empirical Bayesian framework for fMRI‐constrained MEG/EEG source reconstruction , 2010, Human brain mapping.

[19]  Giulio Sandini,et al.  The Role of High Spatial Frequencies in Face Perception , 2013, Perception.

[20]  Maurizio Codispoti,et al.  Spatial frequencies and emotional perception , 2013, Reviews in the neurosciences.

[21]  Wei Wang,et al.  Revealing Detail along the Visual Hierarchy: Neural Clustering Preserves Acuity from V1 to V4 , 2018, Neuron.

[22]  A. Oliva,et al.  From Blobs to Boundary Edges: Evidence for Time- and Spatial-Scale-Dependent Scene Recognition , 1994 .

[23]  Isabel Gauthier,et al.  Holistic Processing Predicts Face Recognition , 2011, Psychological science.

[24]  J. Sergent.  About face: left-hemisphere involvement in processing physiognomies. , 1982, Journal of experimental psychology. Human perception and performance.

[25]  Dariu Gavrila,et al.  Real-time object detection for "smart" vehicles , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[26]  P. O. Bishop,et al.  Spatial vision. , 1971, Annual review of psychology.

[27]  Shiguang Shan,et al.  Coarse-to-Fine Auto-Encoder Networks (CFAN) for Real-Time Face Alignment , 2014, ECCV.

[28]  Bruno Rossion,et al.  Faces are "spatial"--holistic face perception is supported by low spatial frequencies. , 2006, Journal of experimental psychology. Human perception and performance.

[29]  J. Haxby,et al.  The distributed human neural system for face perception , 2000, Trends in Cognitive Sciences.

[30]  P. H. Schiller,et al.  Spatial frequency and orientation tuning dynamics in area V1 , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[31]  M. Eimer.  The Face-Sensitive N170 Component of the Event-Related Brain Potential , 2011 .

[32]  Nathalie Guyader,et al.  Rapid scene categorization: Role of spatial frequency order, accumulation mode and luminance contrast , 2015, Vision Research.

[33]  Martial Mermillod,et al.  Effect of temporal constraints on hemispheric asymmetries during spatial frequency processing , 2006, Brain and Cognition.

[34]  Robert Oostenveld,et al.  FieldTrip: Open Source Software for Advanced Analysis of MEG, EEG, and Invasive Electrophysiological Data , 2010, Comput. Intell. Neurosci..

[35]  H. Halit,et al.  Is high-spatial frequency information used in the early stages of face detection? , 2006, Brain Research.

[36]  M. Bar.  Visual objects in context , 2004, Nature Reviews Neuroscience.

[37]  Mark W. Woolrich,et al.  A symmetric multivariate leakage correction for MEG connectomes , 2015, NeuroImage.

[38]  Yuning Jiang,et al.  Extensive Facial Landmark Localization with Coarse-to-Fine Convolutional Network Cascade , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[39]  H. Hughes,et al.  Global Precedence, Spatial Frequency Channels, and the Statistics of Natural Images , 1996, Journal of Cognitive Neuroscience.

[40]  J R Lishman,et al.  Temporal Integration of Spatially Filtered Visual Images , 1992, Perception.

[41]  Joan Liu-Shuang.  Ultra-coarse, single-glance human face detection in a dynamic visual stream , 2019 .

[42]  A. Oliva,et al.  Dr. Angry and Mr. Smile: when categorization flexibly modifies the perception of faces in rapid visual presentations , 1999, Cognition.

[43]  Chantal Kemner,et al.  Is the early modulation of brain activity by fearful facial expressions primarily mediated by coarse low spatial frequency information? , 2009, Journal of vision.

[44]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[45]  R. Näsänen,et al.  Utilisation of spatial frequency information in face search , 2003, Vision Research.

[46]  Richard N. Henson,et al.  Adaptive cortical parcellations for source reconstructed EEG/MEG connectomes , 2017, NeuroImage.

[47]  Changsong Zhou,et al.  Reconstructing ERP amplitude effects after compensating for trial-to-trial latency jitter: A solution based on a novel application of residue iteration decomposition. , 2016, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[48]  Valérie Goffaux,et al.  Spatial interactions in upright and inverted faces: Re-exploration of spatial scale influence , 2009, Vision Research.

[49]  Sheng Li,et al.  The neural signature of spatial frequency-based information integration in scene perception , 2013, Experimental Brain Research.

[50]  R. Näsänen.  Spatial frequency bandwidth used in the recognition of facial images , 1999, Vision Research.

[51]  O. Pascalis,et al.  Plasticity of face processing in infancy. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[52]  Geoffrey M. Boynton,et al.  Efficient Design of Event-Related fMRI Experiments Using M-Sequences , 2002, NeuroImage.

[53]  A. Young,et al.  Understanding face recognition. , 1986, British journal of psychology.

[54]  Kendrick N Kay,et al.  Bottom-up and top-down computations in word- and face-selective cortex , 2017, eLife.

[55]  R. Watt.  Scanning from coarse to fine spatial scales in the human visual system after the onset of a stimulus. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[56]  Vinh Thai Nguyen,et al.  The superior temporal sulcus and the N170 during face processing: Single trial analysis of concurrent EEG–fMRI , 2014, NeuroImage.

[57]  D. M. Parker,et al.  Perceptual latency and spatial frequency , 1987, Vision Research.

[58]  M. Farah,et al.  What is "special" about face perception? , 1998, Psychological review.

[59]  E. Halgren,et al.  Top-down facilitation of visual recognition. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[60]  Caspar M. Schwiedrzik,et al.  High-Level Prediction Signals in a Low-Level Area of the Macaque Face-Processing Hierarchy , 2017, Neuron.