Cortical dynamics of three-dimensional surface perception: Binocular and half-occluded scenic images

Abstract Previous models of stereopsis have concentrated on the task of binocularly matching left and right eye primitives uniquely. A disparity smoothness constraint is often invoked to limit the number of possible matches. These approaches neglect the fact that surface discontinuities are both abundant in natural everyday scenes, and provide a useful cue for scene segmentation. da Vinci stereopsis refers to the more general problem of dealing with surface discontinuities and their associated unmatched monocular regions within binocular scenes. This study develops a mathematical realization of a neural network theory of biological vision, called FACADE theory, that shows how early cortical stereopsis processes are related to later cortical processes of three-dimensional surface representation. The mathematical model demonstrates through computer simulation how the visual cortex may generate three-dimensional boundary segmentations and use them to control filling-in of three-dimensional surface properties in response to visual scenes. Model mechanisms correctly match disparate binocular regions while filling-in monocular regions with the correct depth within a binocularly viewed scene. This achievement required the introduction of a new multiscale binocular filter for stereo matching which clarifies how cortical complex cells match image contours of like contrast polarity, while pooling signals from opposite contrast polarities. The filter also suggests how false binocular matches and unmatched monocular cues are automatically handled across multiple spatial scales. Pooling of signals from even- and odd-symmetric simple cells at complex cells helps to eliminate spurious activity peaks in matchable signals. Later stages of cortical processing by the blob and interblob streams, including refined models of cooperative boundary grouping and reciprocal stream interactions between boundary and surface representations, are modeled to provide a complete simulation of the da Vinci stereopsis percept.

[1]  S. Grossberg,et al.  Self-Organization of Binocular Disparity Tuning by Reciprocal Corticogeniculate Interactions , 1998, Journal of Cognitive Neuroscience.

[2]  K. Nakayama,et al.  Toward a general theory of stereopsis: binocular matching, occluding contours, and fusion. , 1994, Psychological review.

[3]  Stephen Grossberg,et al.  Contour Enhancement, Short Term Memory, and Constancies in Reverberating Neural Networks , 1973 .

[4]  D. Pollen,et al.  Spatial computation performed by simple and complex cells in the visual cortex of the cat , 1982, Vision Research.

[5]  C. Shatz.  The developing brain. , 1992, Scientific American.

[6]  B. Julesz,et al.  Early visual perception. , 1981, Annual review of psychology.

[7]  D H HUBEL,et al.  RECEPTIVE FIELDS AND FUNCTIONAL ARCHITECTURE IN TWO NONSTRIATE VISUAL AREAS (18 AND 19) OF THE CAT. , 1965, Journal of neurophysiology.

[8]  David Mumford,et al.  A Bayesian treatment of the stereo correspondence problem using half-occluded regions , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  S Grossberg,et al.  Neural dynamics of brightness perception: Features, boundaries, diffusion, and resonance , 1984, Perception & Psychophysics.

[10]  Lloyd Kaufman,et al.  Sight and mind , 1974 .

[11]  C. Tyler,et al.  Failure of rivalry at low contrast: Evidence of a suprathreshold binocular summation process , 1992, Vision Research.

[12]  T. Poggio,et al.  A computational theory of human stereo vision , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[13]  Suzanne P. McKee,et al.  Fusional suppression in normal and stereoanomalous observers , 1993, Vision Research.

[14]  S. McKee,et al.  The imprecision of stereopsis , 1990, Vision Research.

[15]  A. Hodgkin.  The conduction of the nervous impulse , 1964 .

[16]  S. McKee,et al.  Stereo matching precedes dichoptic masking , 1994, Vision Research.

[17]  W. Gogel,et al.  EQUIDISTANCE TENDENCY AND ITS CONSEQUENCES. , 1965, Psychological bulletin.

[18]  Stephen Grossberg,et al.  Synthetic aperture radar processing by a multiple scale neural system for boundary and surface representation , 1995, Neural Networks.

[19]  C. Tyler.  Spatial organization of binocular disparity sensitivity , 1975, Vision Research.

[20]  Hermann von Helmholtz,et al.  Treatise on Physiological Optics , 1962 .

[21]  S Grossberg,et al.  3-D vision and figure-ground separation by visual cortex , 2010, Perception & psychophysics.

[22]  S. Zeki.  Colour coding in the cerebral cortex: The reaction of cells in monkey visual cortex to wavelengths and colours , 1983, Neuroscience.

[23]  Leslie G. Ungerleider,et al.  Contour, color and shape analysis beyond the striate cortex , 1985, Vision Research.

[24]  S. Grossberg.  Cortical dynamics of three-dimensional figure-ground perception of two-dimensional pictures. , 1997, Psychological review.

[25]  P. O. Bishop,et al.  Dimensions and properties of end-zone inhibitory areas in receptive fields of hypercomplex cells in cat striate cortex. , 1979, Journal of neurophysiology.

[26]  S. Grossberg,et al.  Neural dynamics of form perception: boundary completion, illusory figures, and neon color spreading. , 1985 .

[27]  S. Grossberg.  Outline of A Theory of Brightness, Color, and form Perception , 1984 .

[28]  S Grossberg,et al.  Cortical dynamics of three-dimensional form, color, and brightness perception: II. Binocular theory , 1988, Perception & psychophysics.

[29]  Michael S. Landy,et al.  Computational Issues in Solving the Stereo Correspondence ProblemComputational Issues in Solving the Stereo Correspondence Problem , 1991 .

[30]  Julie M. Harris,et al.  Independent neural mechanisms for bright and dark information in binocular stereopsis , 1995, Nature.

[31]  W. Singer,et al.  Selection of intrinsic horizontal connections in the visual cortex by correlated neuronal activity. , 1992, Science.

[32]  C. Schor,et al.  Disparity range for local stereopsis as a function of luminance spatial frequency , 1983, Vision Research.

[33]  Heiko Neumann,et al.  A Contrast- and Luminance-driven Multiscale Network Model of Brightness Perception , 1995, Vision Research.

[34]  Shinsuke Shimojo,et al.  Pre-stereoptic binocular vision in infants , 1986, Vision Research.

[35]  Michael S. Landy,et al.  Computational models of visual processing , 1991 .

[36]  E. Degreef,et al.  Trends in mathematical psychology , 1984 .

[37]  Stephen Grossberg,et al.  Cortical Dynamics of 3-D Figure-Ground Perception of 2-D Pictures , 1995 .

[38]  W Richards,et al.  Local versus global stereopsis: two mechanisms? , 1974, Vision research.

[39]  R. von der Heydt,et al.  Illusory contours and cortical neuron responses. , 1984, Science.

[40]  W. Richards,et al.  Anomalous stereoscopic depth perception. , 1971, Journal of the Optical Society of America.

[41]  Armin Tschermak v. Seysenegg,et al.  Introduction to physiological optics , 1952 .

[42]  B. Julesz,et al.  Independent Spatial-Frequency-Tuned Channels in Binocular Fusion and Rivalry , 1975 .

[43]  S. Levay,et al.  Ocular dominance and disparity coding in cat visual cortex , 1988, Visual Neuroscience.

[44]  G. Poggio,et al.  Stereoscopic mechanisms in monkey visual cortex: binocular correlation and disparity selectivity , 1988, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[45]  Peter H. Schiller,et al.  The ON and OFF channels of the visual system , 1992, Trends in Neurosciences.

[46]  S. Grossberg,et al.  Cortical computation of stereo disparity , 1998, Vision Research.

[47]  D Marr,et al.  Cooperative computation of stereo disparity. , 1976, Science.

[48]  Christopher W. Tyler,et al.  Sensory processing of binocular disparity , 1983 .

[49]  C. Schor,et al.  Binocular sensory fusion is limited by spatial resolution , 1984, Vision Research.

[50]  Heinz Werner,et al.  Dynamics in binocular depth perception , 1937 .

[51]  Rodney Cotterill,et al.  Models of brain function , 1989 .

[52]  Alan N. Gove,et al.  Brightness perception, illusory contours, and corticogeniculate feedback , 1995, Visual Neuroscience.

[53]  Carlos Acuña,et al.  Cell responses to vertical and horizontal retinal disparities in the monkey visual cortex , 1993, Neuroscience Letters.

[54]  S. Grossberg,et al.  Neural dynamics of 1-D and 2-D brightness perception: A unified model of classical and recent phenomena , 1988, Perception & psychophysics.

[55]  Ennio Mingolla,et al.  Neural dynamics of perceptual grouping: Textures, boundaries, and emergent segmentations , 1985 .

[56]  Shinsuke Shimojo,et al.  Da vinci stereopsis: Depth and subjective occluding contours from unpaired image points , 1990, Vision Research.

[57]  D. C. Van Essen,et al.  Concurrent processing streams in monkey visual cortex , 1988, Trends in Neurosciences.

[58]  J. Beck,et al.  Contrast and spatial variables in texture segregation: Testing a simple spatial-frequency channels model , 1989, Perception & psychophysics.

[59]  G Sperling,et al.  Two motion perception mechanisms revealed through distance-driven reversal of apparent motion. , 1989, Proceedings of the National Academy of Sciences of the United States of America.

[60]  S. Zeki.  Colour coding in the cerebral cortex: The responses of wavelength-selective and colour-coded cells in monkey visual cortex to changes in wavelength composition , 1983, Neuroscience.

[61]  Stephen Grossberg,et al.  Neural dynamics of surface perception: Boundary webs, illuminants, and shape-from-shading , 1987, Comput. Vis. Graph. Image Process..

[62]  Christopher W. Tyler,et al.  Spatio-temporal properties of Panum's fusional area , 1981, Vision Research.

[63]  L C Katz,et al.  Development of local circuits in mammalian visual cortex. , 1992, Annual review of neuroscience.

[64]  Lawrence K. Cormack,et al.  Disparity tuning in mechanisms of human stereopsis , 1992, Vision Research.

[65]  D. Pollen,et al.  Phase relationships between adjacent simple cells in the visual cortex. , 1981, Science.

[66]  J. J. KULIKOWSKI,et al.  Limit of single vision in stereopsis depends on contour sharpness , 1978, Nature.

[67]  Stephen Grossberg.  Boundary, Brightness, and Depth Interactions During Preattentive Representation and Attentive Recognition of Figure and Ground , 1993 .