The Gaussian derivative model for spatial-temporal vision: I. Cortical model.

How do we see the motion of objects as well as their shapes? The Gaussian Derivative (GD) spatial model is extended to time to help answer this question. The GD spatio-temporal model requires only two numbers to describe the complete three-dimensional space-time shapes of individual receptive fields in primate visual cortex. These two numbers are the derivative numbers along the respective spatial and temporal principal axes of a given receptive field. Nine transformation parameters allow for a standard geometric association of these intrinsic axes with the extrinsic environment. The GD spatio-temporal model describes in one framework the following properties of primate simple cell fields: motion properties, number of lobes in space-time, spatial orientation. location, and size. A discrete difference-of-offset-Gaussians (DOOG) model provides a plausible physiological mechanism to form GD-like model fields in both space and time. The GD model hypothesizes that receptive fields at the first stage of processing in the visual cortex approximate 'derivative analyzers' that estimate local spatial and temporal derivatives of the intensity profile in the visual environment. The receptive fields as modeled provide operators that can allow later stages of processing in either a biological or machine vision system to estimate the motion as well as the shapes of objects in the environment.

[1]  O. Reiser,et al.  Principles Of Gestalt Psychology , 1936 .

[2]  W. Reichardt,et al.  Autocorrelation, a principle for the evaluation of sensory information by the central nervous system , 1961 .

[3]  Jay Enoch,et al.  Mach Bands: Quantitative Studies on Neural Networks in the Retina , 1966 .

[4]  C. Enroth-Cugell,et al.  The contrast sensitivity of retinal ganglion cells of the cat , 1966, The Journal of physiology.

[5]  J. Robson,et al.  Application of fourier analysis to the visibility of gratings , 1968, The Journal of physiology.

[6]  I.D.G. Macleod,et al.  Comments on "Techniques for edge detection" , 1972 .

[7]  O. Braddick.  A short-range process in apparent motion. , 1974, Vision research.

[8]  F M de Monasterio,et al.  Center and surround mechanisms of opponent-color X and Y ganglion cells of retina of macaques. , 1978, Journal of neurophysiology.

[9]  H. Bülthoff,et al.  Analogous motion illusion in man and fly , 1979, Nature.

[10]  D. Rose.  Mechanisms underlying the receptive field properties of neurons in cat visual cortex , 1979, Vision Research.

[11]  Claude L. Fennema,et al.  Velocity determination in scenes containing several moving objects , 1979 .

[12]  O J Braddick,et al.  Low-level and high-level processes in apparent motion. , 1980, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[13]  S Marcelja,et al.  Mathematical description of the responses of simple cortical cells. , 1980, Journal of the Optical Society of America.

[14]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[15]  D Marr,et al.  Directional selectivity and its use in early visual processing , 1981, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[16]  P. Thompson.  Perceived rate of movement depends on contrast , 1982, Vision Research.

[17]  E. Adelson,et al.  Phenomenal coherence of moving visual patterns , 1982, Nature.

[18]  R. Weale.  Vision. A Computational Investigation Into the Human Representation and Processing of Visual Information. David Marr , 1983 .

[19]  Daniel A. Pollen,et al.  Visual cortical neurons as localized spatial frequency filters , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[20]  J. Canny.  Finding Edges and Lines in Images , 1983 .

[21]  Alex Pentland,et al.  Local Shading Analysis , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  George Mather,et al.  Luminance change generates apparent movement: Implications for models of directional specificity in the human visual system , 1984, Vision Research.

[23]  J. van Santen,et al.  Temporal covariance model of human motion perception. , 1984, Journal of the Optical Society of America. A, Optics and image science.

[24]  Ellen C. Hildreth,et al.  Measurement of Visual Motion , 1984 .

[25]  P Heggelund,et al.  Spatial summation in subregions of simple‐cell receptive fields in cat striate cortex as a function of slit length. , 1984, The Journal of physiology.

[26]  D. Pollen,et al.  Spatial and temporal frequency selectivity of neurones in visual cortical areas V1 and V2 of the macaque monkey. , 1985, The Journal of physiology.

[27]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[28]  A J Ahumada,et al.  Model of human visual-motion sensing. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[29]  Apparent motion from luminance change: Further comments on candidate mechanisms , 1985, Vision Research.

[30]  R. Young.  GAUSSIAN DERIVATIVE THEORY OF SPATIAL VISION: ANALYSIS OF CORTICAL CELL RECEPTIVE FIELD LINE-WEIGHTING PROFILES. , 1985 .

[31]  R. L. de Valois,et al.  Relationship between spatial-frequency and orientation tuning of striate-cortex cells. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[32]  Andrew M. Derrington,et al.  Apparent motion from luminance change: Sequence discriminators see it too , 1985, Vision Research.

[33]  Richard A. Young,et al.  The Gaussian Derivative Model for Machine and Biological Image Processing , 1985 .

[34]  J. van Santen,et al.  Elaborated Reichardt detectors. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[35]  Richard A. Young,et al.  GAUSSIAN DERIVATIVE MODEL FOR MACHINE VISION: VISUAL CORTEX SIMULATION. , 1986 .

[36]  P. Heggelund.  Quantitative studies of enhancement and suppression zones in the receptive field of simple cells in cat striate cortex. , 1986, The Journal of physiology.

[37]  Alex Pentland,et al.  Shading into Texture , 1984, Artif. Intell..

[38]  Ramesh C. Jain,et al.  Invariant surface characteristics for 3D object recognition in range images , 1985, Comput. Vis. Graph. Image Process..

[39]  Richard A. Young,et al.  SIMULATION OF HUMAN RETINAL FUNCTION WITH THE GAUSSIAN DERIVATIVE MODEL. , 1986 .

[40]  P Heggelund,et al.  Quantitative studies of the discharge fields of single cells in cat striate cortex. , 1986, The Journal of physiology.

[41]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  斎藤 秀昭,et al.  Integration of Direction Signals of Image Motion in the Superior Temporal Sulcus of the Macaque Monkey , 1987 .

[43]  C. Koch,et al.  The analysis of visual motion: from computational theory to neuronal mechanisms. , 1986, Annual review of neuroscience.

[44]  R. Shapley,et al.  Linear mechanism of orientation tuning in the retina and lateral geniculate nucleus of the cat. , 1987, Journal of neurophysiology.

[45]  R A Young,et al.  The Gaussian derivative model for spatial vision: I. Retinal mechanisms. , 1988, Spatial vision.

[46]  A. Parker,et al.  Two-dimensional spatial structure of receptive fields in monkey striate cortex. , 1988, Journal of the Optical Society of America. A, Optics and image science.

[47]  K. Nakayama,et al.  The aperture problem—I. Perception of nonrigidity and motion direction in translating sinusoidal lines , 1988, Vision Research.

[48]  D. Ferster.  Spatially opponent excitation and inhibition in simple cells of the cat visual cortex , 1988, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[49]  H. Spitzer,et al.  Complex-cell receptive field models , 1988, Progress in Neurobiology.

[50]  F. Kingdom,et al.  Border effects on brightness: a review of findings, models and issues. , 1988, Spatial vision.

[51]  Jitendra Malik,et al.  A computational model of texture segmentation , 1989, Proceedings CVPR '89: IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[52]  R. Marrocco,et al.  Predictions about chromatic receptive fields assuming random cone connections. , 1989, Journal of theoretical biology.

[53]  C Koch,et al.  Computing motion in the primate's visual system. , 1989, The Journal of experimental biology.

[54]  Jitendra Malik,et al.  A computational model of texture perception , 1989 .

[55]  Jonathan A. Marshall,et al.  Self-organizing neural networks for perception of visual motion , 1990, Neural Networks.

[56]  B Moulden,et al.  Light-dark asymmetries in the Craik-Cornsweet-O'Brien illusion and a new model of brightness coding. , 1990, Spatial vision.

[57]  J Zhang,et al.  Structure of visual perception. , 1990, Proceedings of the National Academy of Sciences of the United States of America.

[58]  P Perona,et al.  Preattentive texture discrimination with early vision mechanisms. , 1990, Journal of the Optical Society of America. A, Optics and image science.

[59]  K. H. Britten,et al.  Neuronal mechanisms of motion perception. , 1990, Cold Spring Harbor symposia on quantitative biology.

[60]  A. L. Humphrey,et al.  Spatial and temporal response properties of lagged and nonlagged cells in cat lateral geniculate nucleus. , 1990, Journal of neurophysiology.

[61]  D G Stork,et al.  Do Gabor functions provide appropriate descriptions of visual cortical receptive fields? , 1990, Journal of the Optical Society of America. A, Optics and image science.

[62]  Jan J. Koenderink,et al.  Solid shape , 1990 .

[63]  M V Srinivasan,et al.  Neural gradient models for the measurement of image velocity , 1990, Visual Neuroscience.

[64]  Y. Chino,et al.  Orientation bias of neurons in the lateral geniculate nucleus of macaque monkeys , 1990, Visual Neuroscience.

[65]  David J. C. MacKay,et al.  Analysis of Linsker's Simulations of Hebbian Rules , 1990, Neural Computation.

[66]  D. Watt.  Visual Processing: Computational Psychophysical and Cognitive Research , 1990 .

[67]  R. Shapley,et al.  Fine structure of receptive-field centers of X and Y cells of the cat , 1991, Visual Neuroscience.

[68]  Jian Yang,et al.  Harmonic oscillator model of early visual image processing , 1991, Other Conferences.

[69]  Richard A. Young,et al.  Oh say, can you see? The physiology of vision , 1991, Electronic Imaging.

[70]  Stanley A. Klein,et al.  "Perfect" displays and "perfect" image compression in space and time , 1991, Electronic Imaging.

[71]  Complicated substructure from simple circularly symmetric Gaussian processes within the centers of goldfish ganglion cell receptive fields. , 1991, Visual neuroscience.

[72]  M. Landy,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[73]  James T. Todd,et al.  The perception of globally coherent motion , 1992, Vision Research.

[74]  E. Adelson,et al.  Directionally selective complex cells and the computation of motion energy in cat visual cortex , 1992, Vision Research.

[75]  Max A. Viergever,et al.  Scale and the differential structure of images , 1992, Image Vis. Comput..

[76]  A neural network model for textural segmentation , 1992 .

[77]  David R. Badcock,et al.  Analysis of the motion of 2-dimensional patterns: Evidence for a second-order process , 1992, Vision Research.

[78]  D. Pollen,et al.  Interneuronal interaction between members of quadrature phase and anti-phase pairs in the cat's visual cortex , 1992, Vision Research.

[79]  David R. Badcock,et al.  Two-stage analysis of the motion of 2-dimensional patterns, what is the first stage? , 1992, Vision Research.

[80]  P. Thompson,et al.  Human speed perception is contrast dependent , 1992, Vision Research.

[81]  Lowell D. Jacobson,et al.  Structural testing of multi-input linear—nonlinear cascade models for cells in macaque striate cortex , 1993, Vision Research.

[82]  Jian Yang,et al.  Bottom-up visual image processing probed with weighted hermite polynomials , 1995, Neural Networks.

[83]  Edward H. Adelson,et al.  The extraction of Spatio-temporal Energy in Human and Machine Vision , 1997 .

[84]  M. Georgeson,et al.  Perceived Location of Bars and Edges in One-dimensional Images: Computational Models and Human Vision , 1997, Vision Research.

[85]  M. Livingstone,et al.  Mechanisms of Direction Selectivity in Macaque V1 , 1998, Neuron.

[86]  Layne T. Watson,et al.  A Gaussian derivative based version of JPEG for image compression and decompression , 1995, IEEE Trans. Image Process..

[87]  Representing contrast detection as an eigenvalue problem. , 1999, Spatial vision.

[88]  R. L. Valois,et al.  Spatial and temporal receptive fields of geniculate and cortical cells and directional selectivity , 2000, Vision Research.

[89]  Nicholas E Scott-Samuel,et al.  Spatial resolution and receptive field height of motion sensors in human vision , 2000, Vision Research.

[90]  Ronald M. Lesperance,et al.  The Gaussian derivative model for spatial-temporal vision: II. Cortical data. , 2001, Spatial vision.

[91]  RussLL L. Ds Vnlos,et al.  SPATIAL FREQUENCY SELECTIVITY OF CELLS IN MACAQUE VISUAL CORTEX , 2022 .