Image Processing by the Human Visual System

The processing power of the human visual system is truly awe-inspiring. Hundreds of millions of years of natural selection have led to the evolution of a visual system whose rapidity and accuracy is quite breathtaking. Images flashed on a screen can often be identified on the basis of only 100 to 150 ms of processing (see Thorpe & Imbert, 1989). Furthermore, the number of different objects and scenes that can be identified by the average human is probably well over 100,000. Such levels of performance make the processing capacities of current image processing technology look positively feeble. And the contrast between biological and artificial vision is made even more striking when one takes into account the fact that the components used by biological visual systems, namely neurons, are several orders of magnitude slower than the components used in current electronics: neurons typically operate with a maximum pulse rate of around 100 Hz, whereas even the humble PC uses components that operate with a clock rate of 10 MHz or more.

[1]  H. Intraub,et al.  Presentation rate and the representation of briefly glimpsed pictures in memory. , 1980, Journal of experimental psychology. Human learning and memory.

[2]  J. Frisby.  Seeing: Illusion, Brain and Mind , 1979 .

[3]  H. Intraub.  Rapid conceptual identification of sequentially presented pictures. , 1981 .

[4]  B. Julesz.  Textons, the elements of texture perception, and their interactions , 1981, Nature.

[5]  D. Rose,et al.  Models of the visual cortex , 1985 .

[6]  John E. W. Mayhew,et al.  Psychophysical and Computational Studies Towards a Theory of Human Stereopsis , 1981, Artif. Intell..

[7]  D. Hubel.  Eye, brain, and vision , 1988 .

[8]  Tomaso Poggio.  Early vision: from computational structure to algorithms and parallel hardware , 1986 .

[9]  Patrick K. Simpson,et al.  Artificial Neural Systems: Foundations, Paradigms, Applications, and Implementations , 1990 .

[10]  V. S. Ramachandran,et al.  Perceptual organization in moving patterns , 1983, Nature.

[11]  E. T. Rolls,et al.  Hypothalamic neuronal responses associated with the sight of food , 1976, Brain Research.

[12]  S. Shipp,et al.  The functional logic of cortical connections , 1988, Nature.

[13]  G. Rhodes,et al.  Identification and ratings of caricatures: Implications for mental representations of faces , 1987, Cognitive Psychology.

[14]  S. Zucker,et al.  Endstopped neurons in the visual cortex as a substrate for calculating curvature , 1987, Nature.

[15]  Leslie G. Ungerleider,et al.  Object vision and spatial vision: two cortical pathways , 1983, Trends in Neurosciences.

[16]  Philip D. Wasserman,et al.  Neural computing - theory and practice , 1989 .

[17]  M. Silverman,et al.  Spatial frequency columns in primary visual cortex. , 1981, Science.

[18]  I. Biederman.  Perceiving Real-World Scenes , 1972, Science.

[19]  E. Halgren,et al.  Neural encoding of individual words and faces by the human hippocampus and amygdala , 1988, Nature.

[20]  M. Potter.  Meaning in visual search. , 1975, Science.

[21]  B. Julesz.  A brief outline of the texton theory of human vision , 1984, Trends in Neurosciences.

[22]  B. A. Baldwin,et al.  Cells in temporal cortex of conscious sheep can respond preferentially to the sight of faces. , 1987, Science.

[23]  J. C. Meadows.  Disturbed perception of colours associated with localized cerebral lesions. , 1974, Brain : a journal of neurology.

[24]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[25]  C. Blakemore,et al.  A second neural mechanism of binocular depth discrimination , 1972, The Journal of physiology.

[26]  D. D. Hoffman,et al.  The computation of structure from fixed-axis motion: rigid structures , 1986, Biological Cybernetics.

[27]  Jan J. Koenderink,et al.  Space, Form and Optical Deformations , 1985 .

[28]  R. M. Siegel,et al.  Perception of three-dimensional structure from motion in monkey and man , 1988, Nature.

[29]  M. Smyth.  Cognition In Action , 1987 .

[30]  Barrie J. Frost,et al.  Neural Mechanisms for Detecting Object Motion and Figure-Ground Boundaries, Contrasted with Self-Motion Detecting Systems , 1985 .

[31]  J. Kaas.  The organization of neocortex in mammals: implications for theories of brain function. , 1987, Annual review of psychology.

[32]  Michael A. Arbib,et al.  Vision, brain, and cooperative computation: an overview , 1990 .

[33]  James A. Anderson,et al.  Neurocomputing: Foundations of Research , 1988 .

[34]  T Poggio,et al.  Parallel integration of vision modules. , 1988, Science.

[35]  Alan L. Yuille,et al.  An Extremum Principle for Shape from Contour , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  S. Ullman.  The measurement of visual motion Computational considerations and some neurophysiological implications , 1983, Trends in Neurosciences.

[37]  R. Shepard,et al.  Mental Rotation of Three-Dimensional Objects , 1971, Science.

[38]  J. Gibson.  The Senses Considered As Perceptual Systems , 1967 .

[39]  E. Adelson,et al.  Early vision and texture perception , 1988, Nature.

[40]  Kathryn T. Spoehr,et al.  Visual Information Processing , 1982 .

[41]  Ennio Mingolla,et al.  Neural dynamics of perceptual grouping: Textures, boundaries, and emergent segmentations , 1985 .

[42]  James T. Todd,et al.  The Analysis of Three-Dimensional Structure from Moving Images , 1985 .

[43]  V. Ramachandran,et al.  The perception of apparent motion. , 1986, Scientific American.

[44]  T. Poggio.  Vision by man and machine. , 1984, Scientific American.

[45]  P. T. Fox,et al.  Positron emission tomographic studies of the cortical anatomy of single-word processing , 1988, Nature.

[46]  H. Barlow,et al.  A model for the economical encoding of the visual image in cerebral cortex , 2004, Biological Cybernetics.

[47]  Joel S. Warm,et al.  Psychology of Perception , 1957 .

[48]  B. Julesz,et al.  Early visual perception. , 1981, Annual review of psychology.

[49]  Kunihiko Fukushima,et al.  Cognitron: A self-organizing multilayered neural network , 1975, Biological Cybernetics.

[50]  J. Allman,et al.  Stimulus specific responses from beyond the classical receptive field: neurophysiological mechanisms for local-global comparisons in visual neurons. , 1985, Annual review of neuroscience.

[51]  J. Robson,et al.  Application of fourier analysis to the visibility of gratings , 1968, The Journal of physiology.

[52]  C. Blakemore,et al.  The neural mechanism of binocular depth discrimination , 1967, The Journal of physiology.

[53]  John Ross,et al.  Visual processing of motion , 1986, Trends in Neurosciences.

[54]  P. H. Schiller.  The central visual system , 1986, Vision Research.

[55]  T. Poggio,et al.  The analysis of stereopsis. , 1984, Annual review of neuroscience.

[56]  I. Biederman.  Recognition-by-components: a theory of human image understanding. , 1987, Psychological review.

[57]  Glyn W. Humphreys,et al.  Normal and pathological processes in visual object constancy. , 1987 .

[58]  E. Land.  The retinex theory of color vision. , 1977, Scientific American.

[59]  K. Nakayama,et al.  Single visual neurons code opposing motion independent of direction. , 1983, Science.

[60]  A. Gilchrist.  The perception of surface blacks and whites. , 1979, Scientific American.

[61]  M. Cynader,et al.  The visual perception of motion in depth. , 1979, Scientific American.

[62]  C. Koch,et al.  Neuronal connections underlying orientation selectivity in cat visual cortex , 1987, Trends in Neurosciences.

[63]  V. Ramachandran,et al.  The Neurobiology of Perception , 1985, Perception.

[64]  T. Poggio,et al.  The synaptic veto mechanism: does it underlie direction and orientation selectivity in the visual cortex , 1985 .

[65]  E. Warrington,et al.  Evidence for modality-specific meaning systems in the brain , 1988, Nature.

[66]  Jerome A. Feldman,et al.  Connectionist Models and Their Properties , 1982, Cogn. Sci..

[67]  H Goodglass,et al.  Disorders of naming following brain injury. , 1980, American scientist.

[68]  John M. Wilding.  Perception, from sense to object , 1982 .

[69]  John H. R. Maunsell,et al.  Visual processing in monkey extrastriate cortex. , 1987, Annual review of neuroscience.

[70]  E. Rolls,et al.  The latency of activation of neurones in the lateral hypothalamus and substantia innominata during feeding in the monkey , 1979, Brain Research.

[71]  T. Poggio,et al.  Synapses that compute motion. , 1987, Scientific American.

[72]  D. Jameson,et al.  Essay concerning color constancy. , 1989, Annual review of psychology.

[73]  H. C. Longuet-Higgins.  The reconstruction of a plane surface from two perspective projections , 1986, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[74]  M. Potter,et al.  Time to understand pictures and words , 1975, Nature.

[75]  Geoffrey E. Hinton,et al.  Parallel visual computation , 1983, Nature.

[76]  John H. R. Maunsell,et al.  Hierarchical organization and functional streams in the visual cortex , 1983, Trends in Neurosciences.

[77]  H. C. Longuet-Higgins,et al.  A computer algorithm for reconstructing a scene from two projections , 1981, Nature.

[78]  D. V. van Essen,et al.  Retinotopic organization of human visual cortex mapped with positron- emission tomography , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[79]  D. Marr,et al.  Representation and recognition of the spatial organization of three-dimensional shapes , 1978, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[80]  Shimon Ullman.  Artificial intelligence and the neurosciences , 1986, Trends in Neurosciences.

[81]  M. Posner,et al.  Localization of cognitive operations in the human brain. , 1988, Science.

[82]  Jerome A. Feldman,et al.  A functional model of vision and space , 1987 .

[83]  Patrick Henry Winston,et al.  The psychology of computer vision , 1976, Pattern Recognit..

[84]  T. Wiesel,et al.  Functional architecture of cortex revealed by optical imaging of intrinsic signals , 1986, Nature.

[85]  Ken Nakayama,et al.  Biological image motion processing: A review , 1985, Vision Research.

[86]  P. Jolicoeur.  The time to name disoriented natural objects , 1985, Memory & cognition.

[87]  James L. McClelland,et al.  Parallel Distributed Processing: Explorations in the Microstructure of Cognition : Psychological and Biological Models , 1986 .

[88]  S. Kosslyn,et al.  Components of high-level vision: A cognitive neuroscience analysis and accounts of neurological syndromes , 1990, Cognition.

[89]  Katsushi Ikeuchi,et al.  Numerical Shape from Shading and Occluding Boundaries , 1981, Artif. Intell..

[90]  A. J. Mistlin,et al.  Visual neurones responsive to faces , 1987, Trends in Neurosciences.

[91]  B. Julesz.  Foundations of Cyclopean Perception , 1971 .

[92]  W E Grimson,et al.  A computational theory of visual surface interpolation. , 1982, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[93]  Eva Wong,et al.  CHAPTER 2 – Figure–Ground Organization and the Spatial and Temporal Responses of the Visual System , 1986 .

[94]  Harry G. Barrow,et al.  Interpreting Line Drawings as Three-Dimensional Surfaces , 1980, Artif. Intell..

[95]  D Marr,et al.  Directional selectivity and its use in early visual processing , 1981, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[96]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: I. An account of basic findings. , 1981 .

[97]  tephen E. Palmer.  The effects of contextual scenes on the identification of objects , 1975, Memory & cognition.

[98]  Azriel Rosenfeld,et al.  Human and machine vision II , 1986 .

[99]  C. Koch,et al.  The analysis of visual motion: from computational theory to neuronal mechanisms. , 1986, Annual review of neuroscience.

[100]  E. Adelson,et al.  The analysis of moving visual patterns , 1985 .

[101]  Eva Wong,et al.  Figure-ground organization affects the early visual processing , 1987 .

[102]  Steven W. Zucker,et al.  The diversity of perceptual grouping , 1987 .

[103]  W. Singer,et al.  Stimulus-specific neuronal oscillations in orientation columns of cat visual cortex. , 1989, Proceedings of the National Academy of Sciences of the United States of America.

[104]  E T Rolls,et al.  Neuronal responses related to visual recognition. , 1982, Brain : a journal of neurology.

[105]  J. Feldman.  Four frames suffice: A provisional model of vision and space , 1985, Behavioral and Brain Sciences.

[106]  R. Wurtz,et al.  Probing visual cortical function with discrete chemical lesions , 1988, Trends in Neurosciences.

[107]  Tomaso Poggio,et al.  Computing texture boundaries from images , 1988, Nature.

[108]  K. Bötzel,et al.  Electric brain potentials evoked by pictures of faces and non-faces: a search for “face-specific” EEG-potentials , 2004, Experimental Brain Research.

[109]  Leo Maurice Hurvich,et al.  Color vision , 1981 .

[110]  R. Shapley,et al.  Cat and monkey retinal ganglion cells and their visual functional roles , 1986, Trends in Neurosciences.

[111]  B Rogers,et al.  Motion Parallax as an Independent Cue for Depth Perception , 1979, Perception.

[112]  Robert M. Douglas,et al.  A time-based stereoscopic depth mechanism in the visual cortex , 1985, Brain Research.

[113]  Janus J. Kulikowski,et al.  Image analysis performed by the visual system: Feature versus Fourier analysis and adaptable filtering. In: Visual Neuroscience. Cambridge Univ. Press , 1986 .

[114]  Geoffrey E. Hinton,et al.  Scene-based and viewer-centered representations for comparing shapes , 1988, Cognition.

[115]  N. Mai,et al.  Selective disturbance of movement vision after bilateral brain damage. , 1983, Brain : a journal of neurology.

[116]  W. Pitts,et al.  What the Frog's Eye Tells the Frog's Brain , 1959, Proceedings of the IRE.

[117]  C Blakemore,et al.  On the existence of neurones in the human visual system selectively sensitive to the orientation and size of retinal images , 1969, The Journal of physiology.

[118]  S. Zeki.  The representation of colours in the cerebral cortex , 1980, Nature.

[119]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[120]  A. Cowey.  Cortical Maps and Visual Perception the Grindley Memorial Lecture* , 1979, The Quarterly journal of experimental psychology.

[121]  S. Pinker,et al.  Visual cognition : An introduction * , 1989 .

[122]  Hans-Hellmut Nagel,et al.  On the Estimation of Optical Flow: Relations between Different Approaches and Some New Results , 1987, Artif. Intell..

[123]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[124]  Oscar Firschein,et al.  Readings in computer vision: issues, problems, principles, and paradigms , 1987 .

[125]  Terrence J. Sejnowski,et al.  Separating figure from ground with a Boltzmann machine , 1990 .

[126]  David C. Burr,et al.  Visual analysis during motion , 1990 .

[127]  Martin A. Fischler,et al.  Computational Stereo , 1982, CSUR.

[128]  J. E. W. Mayhew,et al.  A computational model of binocular depth perception , 1982, Nature.

[129]  R N Shepard,et al.  Path-guided apparent motion. , 1983, Science.

[130]  Daryl T. Lawton,et al.  Computational techniques in motion processing , 1987 .

[131]  Robert J. Baron,et al.  The Cerebral Computer: An Introduction To the Computational Structure of the Human Brain , 1987 .

[132]  D. C. Van Essen,et al.  Concurrent processing streams in monkey visual cortex , 1988, Trends in Neurosciences.

[133]  J E Cutting,et al.  Visual flow and direction of locomotion. , 1985, Science.

[134]  Michael Brady,et al.  Computational Approaches to Image Understanding , 1982, CSUR.

[135]  Lee Dn,et al.  The optic flow field: the foundation of vision. , 1980 .

[136]  H. Wilson.  Psychophysical Evidence for Spatial Channels , 1983 .

[137]  E I Knudsen,et al.  Computational maps in the brain. , 1987, Annual review of neuroscience.

[138]  D. Hubel.  Exploration of the primary visual cortex, 1955–78 , 1982, Nature.

[139]  R. Berndt,et al.  Category-specific naming deficit following cerebral infarction , 1985, Nature.

[140]  V S Ramachandran,et al.  Perceiving shape from shading. , 1988, Scientific American.

[141]  Martin Egelhaaf,et al.  Visual course control in flies relies on neuronal computation of object and background motion , 1988, Trends in Neurosciences.

[142]  Dana H. Ballard,et al.  Cortical connections and parallel processing: Structure and function , 1986, Behavioral and Brain Sciences.

[143]  Martin D. Levine,et al.  Vision in Man and Machine , 1985 .

[144]  Kunihiko Fukushima,et al.  A neural network model for selective attention in visual pattern recognition , 1986, Biological Cybernetics.

[145]  D Regan,et al.  How do we avoid confounding the direction we are looking and the direction we are moving? , 1982, Science.

[146]  E. Warrington.  Neuropsychological studies of object recognition. , 1982, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[147]  G. Blasdel,et al.  Voltage-sensitive dyes reveal a modular organization in monkey striate cortex , 1986, Nature.

[148]  D. Marr,et al.  Representation and recognition of the movements of shapes , 1982, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[149]  G. Johansson.  Visual motion perception. , 1975, Scientific American.

[150]  Andrew C. Sleigh,et al.  Physical and Biological Processing of Images , 1983 .

[151]  S. Grossberg.  Cortical dynamics of three-dimensional form, color, and brightness perception: I. Monocular theory , 1987, Perception & psychophysics.

[152]  Tomaso Poggio,et al.  Synthesizing a color algorithm from examples , 1988 .

[153]  Terrence J. Sejnowski,et al.  Network model of shape-from-shading: neural function arises from both receptive and projective fields , 1988, Nature.

[154]  V. S. Ramachandran,et al.  Perception of shape from shading , 1988, Nature.

[155]  R. Shepard,et al.  Perceptual illusion of rotation of three-dimensional objects. , 1976, Science.

[156]  D Regan,et al.  Visually guided locomotion: psychophysical evidence for a neural mechanism sensitive to flow patterns. , 1979, Science.

[157]  C. G. Phillips,et al.  Localization of function in the cerebral cortex. Past, present and future. , 1984, Brain : a journal of neurology.

[158]  R. Desimone,et al.  Behavioral neurophysiology: insights into seeing and grasping. , 1988, Science.

[159]  A. Kaneko.  Physiology of the retina. , 1979, Annual review of neuroscience.

[160]  F. W. Edridge-Green,et al.  COLOUR-VISION AND COLOUR-BLINDNESS. , 1930 .

[161]  T. Sejnowski,et al.  Perspectives on cognitive neuroscience. , 1988, Science.

[162]  Anne Treisman,et al.  Features and objects in visual processing , 1986 .

[163]  Ellen C. Hildreth,et al.  Measurement of Visual Motion , 1984 .

[164]  Takeo Kanade,et al.  Recovery of the Three-Dimensional Shape of an Object from a Single View , 1981, Artif. Intell..

[165]  R. C. Oldfield.  Things, Words and the Brain* , 1966, The Quarterly journal of experimental psychology.

[166]  J. Fodor.  The Modularity of mind. An essay on faculty psychology , 1986 .

[167]  Whitman A. Richards.  Natural Computation , 1988 .

[168]  T. Poggio,et al.  A computational theory of human stereo vision , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[169]  P. McLeod.  Visual Reaction Time and High-Speed Ball Games , 1987, Perception.

[170]  Feldman,et al.  Connectionist models and parallelism in high-level vision. Technical report , 1985 .

[171]  S. Ullman.  Visual routines , 1984, Cognition.

[172]  Leslie G. Ungerleider,et al.  Contour, color and shape analysis beyond the striate cortex , 1985, Vision Research.

[173]  R. Gregory.  The intelligent eye , 1970 .

[174]  S W Zucker,et al.  Receptive fields and the representation of visual information. , 1986, Human neurobiology.

[175]  A. J. Mistlin,et al.  Visual cells in the temporal cortex sensitive to face view and gaze direction , 1985, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[176]  S. Petersen,et al.  The neurobiology of attention , 1986 .

[177]  Kunihiko Fukushima,et al.  Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position , 1980, Biological Cybernetics.

[178]  Glyn W. Humphreys,et al.  On telling your fruit from your vegetables: a consideration of category-specific deficits after brain damage , 1987, Trends in Neurosciences.

[179]  E. Hildreth.  The computation of the velocity field , 1984, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[180]  D Marr,et al.  Cooperative computation of stereo disparity. , 1976, Science.

[181]  Andrew P. Witkin,et al.  Recovering Surface Shape and Orientation from Texture , 1981, Artif. Intell..