Extraction of Surface-Related Features in a Recurrent Model of V1-V2 Interactions

Background Humans can effortlessly segment surfaces and objects from two-dimensional (2D) images that are projections of the 3D world. The projection from 3D to 2D leads partially to occlusions of surfaces depending on their position in depth and on viewpoint. One way for the human visual system to infer monocular depth cues could be to extract and interpret occlusions. It has been suggested that the perception of contour junctions, in particular T-junctions, may be used as cue for occlusion of opaque surfaces. Furthermore, X-junctions could be used to signal occlusion of transparent surfaces. Methodology/Principal Findings In this contribution, we propose a neural model that suggests how surface-related cues for occlusion can be extracted from a 2D luminance image. The approach is based on feedforward and feedback mechanisms found in visual cortical areas V1 and V2. In a first step, contours are completed over time by generating groupings of like-oriented contrasts. Few iterations of feedforward and feedback processing lead to a stable representation of completed contours and at the same time to a suppression of image noise. In a second step, contour junctions are localized and read out from the distributed representation of boundary groupings. Moreover, surface-related junctions are made explicit such that they are evaluated to interact as to generate surface-segmentations in static images. In addition, we compare our extracted junction signals with a standard computer vision approach for junction detection to demonstrate that our approach outperforms simple feedforward computation-based approaches. Conclusions/Significance A model is proposed that uses feedforward and feedback mechanisms to combine contextually relevant features in order to generate consistent boundary groupings of surfaces. Perceptually important junction configurations are robustly extracted from neural representations to signal cues for occlusion and transparency. Unlike previous proposals which treat localized junction configurations as 2D image features, we link them to mechanisms of apparent surface segregation. As a consequence, we demonstrate how junctions can change their perceptual representation depending on the scene context and the spatial configuration of boundary fragments.

[1]  Hans Wallach.  Über visuell wahrgenommene Bewegungsrichtung , 1935 .

[2]  O. Reiser,et al.  Principles Of Gestalt Psychology , 1936 .

[3]  G. Kanizsa.  Margini Quasi-percettivi in Campi con Stimolazione Omogenea , 1955 .

[4]  D H HUBEL,et al.  RECEPTIVE FIELDS AND FUNCTIONAL ARCHITECTURE IN TWO NONSTRIATE VISUAL AREAS (18 AND 19) OF THE CAT. , 1965, Journal of neurophysiology.

[5]  D. Hubel,et al.  Receptive fields and functional architecture of monkey striate cortex , 1968, The Journal of physiology.

[6]  G. Sperling.  Model of visual adaptation and contrast detection , 1970 .

[7]  F Metelli,et al.  The perception of transparency. , 1974, Scientific American.

[8]  L. Maffei,et al.  The unresponsive regions of visual cortical receptive fields , 1976, Vision Research.

[9]  R. von der Heydt,et al.  Illusory contours and cortical neuron responses. , 1984, Science.

[10]  S. Grossberg,et al.  Neural dynamics of form perception: boundary completion, illusory figures, and neon color spreading. , 1985, Psychological review.

[11]  T. Wiesel,et al.  Relationships between horizontal interactions and functional architecture in cat striate cortex as revealed by cross-correlation analysis , 1986, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[12]  G. Phillips,et al.  Cooperative phenomena in the perception of motion direction. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[13]  John G. Daugman,et al.  Complete discrete 2-D Gabor transforms by neural networks for image analysis and compression , 1988, IEEE Trans. Acoust. Speech Signal Process..

[14]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[15]  R. von der Heydt,et al.  Mechanisms of contour perception in monkey visual cortex. II. Contours bridging gaps , 1989, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[16]  T. Wiesel,et al.  Columnar specificity of intrinsic horizontal and corticocortical connections in cat visual cortex , 1989, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[17]  Reinhard Eckhorn,et al.  Feature Linking via Synchronization among Distributed Assemblies: Simulations of Results from Cat Visual Cortex , 1990, Neural Computation.

[18]  M. Landy,et al.  Transparency and the Cooperative Computation of Scene Attributes , 1991 .

[19]  Michael S. Landy,et al.  Computational models of visual processing , 1991 .

[20]  P. Kellman,et al.  A theory of visual interpolation in object perception , 1991, Cognitive Psychology.

[21]  C. Gilbert,et al.  Synaptic physiology of horizontal connections in the cat's visual cortex , 1991, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[22]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[23]  D. Kersten.  Transparency and the cooperative computation of scene attributes , 1991 .

[24]  D. V. van Essen,et al.  Neuronal responses to static texture patterns in area V1 of the alert macaque monkey. , 1992, Journal of neurophysiology.

[25]  Maggie Shiffrar,et al.  The influence of terminators on motion integration across space , 1992, Vision Research.

[26]  D. Heeger.  Normalization of cell responses in cat striate cortex , 1992, Visual Neuroscience.

[27]  Ennio Mingolla,et al.  The role of edges and line-ends in illusory contour formation , 1993, Vision Research.

[28]  Takeo Watanabe,et al.  Transparent surfaces defined by implicit X junctions , 1993, Vision Research.

[29]  David J. Field,et al.  Contour integration by the human visual system: Evidence for a local “association field” , 1993, Vision Research.

[30]  F. Heitger,et al.  Perception of occluding contours: Neural mechanisms and a computational model , 1993 .

[31]  I. Ohzawa,et al.  Length and width tuning of neurons in the cat's primary visual cortex. , 1994, Journal of neurophysiology.

[32]  Keiji Tanaka,et al.  Neuronal selectivities to complex object features in the ventral visual pathway of the macaque cerebral cortex. , 1994, Journal of neurophysiology.

[33]  M. Carandini,et al.  Summation and division by neurons in primate visual cortex. , 1994, Science.

[34]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.

[35]  H. Jones,et al.  Visual cortical mechanisms detecting focal orientation discontinuities , 1995, Nature.

[36]  T. Sejnowski,et al.  A selection model for motion processing in area MT of primates , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[37]  Alan N. Gove,et al.  Brightness perception, illusory contours, and corticogeniculate feedback , 1995, Visual Neuroscience.

[38]  P A Salin,et al.  Corticocortical connections in the visual system: structure and function. , 1995, Physiological reviews.

[39]  Shinsuke Shimojo,et al.  Visual surface representation: a critical link between lower-level and higher level vision , 1995 .

[40]  C. Gilbert,et al.  Improvement in visual sensitivity by changes in local context: Parallel studies in human observers and in V1 of alert monkeys , 1995, Neuron.

[41]  B. Anderson.  A Theory of Illusory Lightness and Transparency in Monocular and Binocular Images: The Role of Contour Junctions , 1997, Perception.

[42]  Esther Peterhans,et al.  Functional Organization of Area V2 in the Awake Monkey , 1997 .

[43]  D. Heeger,et al.  Comparison of contrast-normalization and threshold models of the responses of simple cells in cat striate cortex , 1997, Visual Neuroscience.

[44]  W Singer,et al.  The Perceptual Grouping Criterion of Colinearity is Reflected by Anisotropies of Connections in the Primary Visual Cortex , 1997, The European journal of neuroscience.

[45]  J. Kaas,et al.  Extrastriate Cortex in Primates , 1997, Cerebral Cortex.

[46]  Edward E. Smith,et al.  An Invitation to cognitive science , 1997 .

[47]  D. Fitzpatrick,et al.  Orientation Selectivity and the Arrangement of Horizontal Connections in Tree Shrew Striate Cortex , 1997, The Journal of Neuroscience.

[48]  C. Koch,et al.  Constraints on cortical and thalamic projections: the no-strong-loops hypothesis , 1998, Nature.

[49]  Rüdiger von der Heydt,et al.  Simulation of neural contour mechanisms: representing anomalous contours , 1998, Image Vis. Comput..

[50]  J. M. Hupé,et al.  Cortical feedback improves discrimination between figure and background by V1, V2 and V3 neurons , 1998, Nature.

[51]  G Westheimer,et al.  Dynamics of spatial summation in primary visual cortex of alert monkeys. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[52]  C. Connor,et al.  Responses to contour features in macaque area V4. , 1999, Journal of neurophysiology.

[53]  Heiko Neumann,et al.  Recurrent V1–V2 interaction in early visual boundary processing , 1999, Biological Cybernetics.

[54]  Z Li,et al.  Pre-attentive segmentation in the primary visual cortex. , 1998, Spatial vision.

[55]  R. von der Heydt,et al.  Coding of Border Ownership in Monkey Visual Cortex , 2000, The Journal of Neuroscience.

[56]  V. Lamme,et al.  The distinct modes of vision offered by feedforward and recurrent processing , 2000, Trends in Neurosciences.

[57]  Stephen Grossberg,et al.  Visual cortical mechanisms of perceptual grouping: interacting layers, networks, columns, and maps , 2000, Neural Networks.

[58]  P Girard,et al.  Feedback connections act on the early part of the responses in monkey visual cortex. , 2001, Journal of neurophysiology.

[59]  G. Baylis,et al.  Shape-coding in IT cells generalizes over contrast and mirror reversal, but not figure-ground reversal , 2001, Nature Neuroscience.

[60]  N Rubin,et al.  The Role of Junctions in Surface Completion and Contour Matching , 2001, Perception.

[61]  Nava Rubin,et al.  Figure and ground in the brain , 2001, Nature Neuroscience.

[62]  Generalization over contrast and mirror reversal, but not figure-ground reversal, in an "edge-based , 2001 .

[63]  R. Shapley,et al.  Visual spatial characterization of macaque V1 neurons. , 2001, Journal of neurophysiology.

[64]  Christopher C. Pack,et al.  Temporal dynamics of a neural solution to the aperture problem in visual area MT of macaque brain , 2001, Nature.

[65]  Heiko Neumann,et al.  Computational Neural Models of Spatial Integration in Perceptual Grouping , 2001 .

[66]  H. Jones,et al.  Surround suppression in primate V1. , 2001, Journal of neurophysiology.

[67]  S. Hochstein,et al.  View from the Top Hierarchies and Reverse Hierarchies in the Visual System , 2002, Neuron.

[68]  Laurent Itti,et al.  CINNIC, a new computational algorithm for the modeling of early visual contour integration in humans , 2003, Neurocomputing.

[69]  Margaret S Livingstone,et al.  End-Stopping and the Aperture Problem Two-Dimensional Motion Signals in Macaque V1 , 2003, Neuron.

[70]  G. Boynton,et al.  Visual Cortex: The Continuing Puzzle of Area V2 , 2004, Current Biology.

[71]  Cordelia Schmid,et al.  Evaluation of Interest Point Detectors , 2000, International Journal of Computer Vision.

[72]  Edward H Adelson,et al.  The geometry of the occluding contour and its effect on motion interpretation. , 2004, Journal of vision.

[73]  Stephen M. Smith,et al.  SUSAN—A New Approach to Low Level Image Processing , 1997, International Journal of Computer Vision.

[74]  Minami Ito,et al.  Representation of Angles Embedded within Contour Stimuli in Area V2 of Macaque Monkeys , 2004, The Journal of Neuroscience.

[75]  Heiko Neumann,et al.  Disambiguating Visual Motion Through Contextual Feedback Modulation , 2004, Neural Computation.

[76]  Heiko Neumann,et al.  Neural Mechanisms for the Robust Representation of Junctions , 2004, Neural Computation.

[77]  Li Zhaoping,et al.  Border Ownership from Intracortical Interactions in Visual Area V2 , 2005, Neuron.

[78]  Randall S. Birnkrant,et al.  Visual search for transparency and opacity: attentional guidance by cue combination? , 2005, Journal of vision.

[79]  H. Neumann,et al.  Neural mechanisms of human texture processing: texture boundary detection and visual search. , 2005, Spatial vision.

[80]  Fred Henrik Hamker,et al.  The emergence of attention by population-based inference and its role in distributed processing and cognitive control of vision , 2005, Comput. Vis. Image Underst..

[81]  M. Young,et al.  Primary visual cortex neurons that contribute to resolve the aperture problem , 2006, Neuroscience.

[82]  A. Sillito,et al.  Always returning: feedback and sensory processing in visual cortex and thalamus , 2006, Trends in Neurosciences.

[83]  D. C. Essen,et al.  Neurons in monkey visual area V2 encode combinations of orientations , 2007, Nature Neuroscience.

[84]  Heiko Neumann,et al.  Globally consistent depth sorting of overlapping 2D surfaces in a model using local recurrent interactions , 2008, Biological Cybernetics.