The What and Why of Binding: Review The Modeler's Perspective

our parlance, mechanisms of organization). Neurosci-ence and neural modeling, on the other hand, have the ambition to find general answers. It is this commitment Christoph von der Malsburg* Institut fü r Neuroinformatik Ruhr-Universitä t Bochum to generality that results in the binding problem being 44780 Bochum a fundamental feature of the neural code. Federal Republic of Germany The Psychological Moment University of Southern California Before discussing the answers to the above questions Los Angeles, California 90089 that are postulated by classical neural networks, it is important to introduce an important parameter. Although not often made explicit, it is important to fix a In attempts to formulate a computational understanding temporal scale T, which we will refer to as the " psychological moment " (in the sense of " short period "). At times of brain function, one of the fundamental concerns is shorter than T, one speaks of mental state or brain state, the data structure by which the brain represents infor-whereas at times greater than T one sees a succession mation. For many decades, a conceptual framework of states or a " state history. " Whereas state history is has dominated the thinking of both brain modelers and subject to conscious scrutiny (that is, it potentially gets neurobiologists. That framework is referred to here as reflected in all modalities—memory, language, etc.), no " classical neural networks. " It is well supported by ex-such conscious analysis is possible below T. State his-perimental data, although it may be incomplete. A char-tory is ignored by most or all models, and the conceptual acterization of this framework will be offered in the next disagreement about the binding issue focuses exclu-section. sively on the definition of state, that is, on times below Difficulties in modeling important functional aspects T. It is difficult to pin down a definite value for T, but a of the brain on the basis of classical neural networks plausible range may be from 50 to 200 msec. Regardless alone have led to the recognition that another, general of the exact value, it is important to realize that the arena mechanism must be invoked to explain brain function. for the discussion of the binding problem is at a time That mechanism I call " binding. " Binding by neural signal scale less than T. synchrony had been mentioned several times in the liter-before it was fully of Brain Function formulated as …

[1]  W. Singer,et al.  Stimulus-dependent synchronization of neuronal responses in the visual cortex of the awake macaque monkey , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[2]  John K. Tsotsos Toward a computational model of visual attention , 1995 .

[3]  J. Wolfe,et al.  Preattentive Object Files: Shapeless Bundles of Basic Features , 1997, Vision Research.

[4]  J. B. Levitt,et al.  Receptive fields and functional architecture of macaque V2. , 1994, Journal of neurophysiology.

[5]  W. Modell On the significance of significant , 1981, The New England journal of medicine.

[6]  F. Varela Resonant cell assemblies: a new approach to cognitive functions and neuronal synchrony. , 1995, Biological research.

[7]  R. Desimone,et al.  Spectral properties of V4 neurons in the macaque , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[8]  M. Margulis,et al.  Temporal integration can readily switch between sublinear and supralinear summation. , 1998, Journal of neurophysiology.

[9]  W. Singer,et al.  Synchronous oscillations in the cat retina , 1999, Vision Research.

[10]  T. Sejnowski,et al.  A selection model for motion processing in area MT of primates , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[11]  J Saarinen,et al.  Target Localisation and Identification in Rapid Visual Search , 1996, Perception.

[12]  R. Desimone,et al.  Competitive Mechanisms Subserve Attention in Macaque Areas V2 and V4 , 1999, The Journal of Neuroscience.

[13]  M. Potter Meaning in visual search. , 1975, Science.

[14]  G. Buzsáki,et al.  Sharp wave-associated high-frequency oscillation (200 Hz) in the intact hippocampus: network and intracellular mechanisms , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[15]  C. Blakemore,et al.  Single-fibre EPSPs in layer 5 of rat visual cortex in vitro. , 1993, Neuroreport.

[16]  Michael C. Mozer,et al.  The perception of multiple objects , 1991 .

[17]  A. C. Yu,et al.  Temporal Hierarchical Control of Singing in Birds , 1996, Science.

[18]  A. Treisman,et al.  Perceiving visually presented objets: recognition, awareness, and modularity , 1998, Current Opinion in Neurobiology.

[19]  D. McCormick,et al.  H-Current Properties of a Neuronal and Network Pacemaker , 1998, Neuron.

[20]  W. Singer,et al.  Selection of intrinsic horizontal connections in the visual cortex by correlated neuronal activity. , 1992, Science.

[21]  J. Donoghue,et al.  Oscillations in local field potentials of the primate motor cortex during voluntary movement. , 1993, Proceedings of the National Academy of Sciences of the United States of America.

[22]  Denis Fize,et al.  Speed of processing in the human visual system , 1996, Nature.

[23]  A. Treisman,et al.  Illusory words: the roles of attention and of top-down constraints in conjoining letters to form words. , 1986, Journal of experimental psychology. Human perception and performance.

[24]  K. Tanaka,et al.  Analysis of motion of the visual field by direction, expansion/contraction, and rotation cells clustered in the dorsal part of the medial superior temporal area of the macaque monkey. , 1989, Journal of neurophysiology.

[25]  Wolf Singer,et al.  Development and Plasticity of Cortical Processing Architectures , 1995, Science.

[26]  Laurenz Wiskott,et al.  Face recognition by dynamic link matching , 1996 .

[27]  A Treisman,et al.  Feature binding, attention and object perception. , 1998, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[28]  E. Fetz,et al.  Synchronization of neurons during local field potential oscillations in sensorimotor cortex of awake monkeys. , 1996, Journal of neurophysiology.

[29]  Ken Nakayama,et al.  Serial and parallel processing of visual feature conjunctions , 1986, Nature.

[30]  A. Aertsen,et al.  Spike synchronization and rate modulation differentially involved in motor cortical function. , 1997, Science.

[31]  M. Young,et al.  On oscillating neuronal responses in the visual cortex of the monkey. , 1992, Journal of neurophysiology.

[32]  J. Saarinen Localization and discrimination of “pop-out” targets , 1996, Vision Research.

[33]  D. V. Essen,et al.  Neural mechanisms of form and motion processing in the primate visual system , 1994, Neuron.

[34]  John K. Tsotsos Analyzing vision at the complexity level , 1990, Behavioral and Brain Sciences.

[35]  I. Ohzawa,et al.  The effects of contrast on visual orientation and spatial frequency discrimination: a comparison of single cells and behavior. , 1987, Journal of neurophysiology.

[36]  A. M. Smith,et al.  A century after cajal. , 1993, Science.

[37]  Marvin M. Chun,et al.  Making use of texton gradients: visual search and perceptual grouping exploit the same parallel processes in different ways , 1995 .

[38]  N. Logothetis Single units and conscious vision. , 1998, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[39]  M. Mozer Letter migration in word perception. , 1983, Journal of experimental psychology. Human perception and performance.

[40]  M. Young,et al.  Sparse population coding of faces in the inferotemporal cortex. , 1992, Science.

[41]  Ehud Zohary,et al.  Correlated neuronal discharge rate and its implications for psychophysical performance , 1994, Nature.

[42]  P A Salin,et al.  Corticocortical connections in the visual system: structure and function. , 1995, Physiological reviews.

[43]  I. Rock,et al.  GROUPING BASED ON PHENOMENAL PROXIMITY. , 1964, Journal of experimental psychology.

[44]  A. Treisman,et al.  The Interaction of Spatial and Object Pathways: Evidence from Balint's Syndrome , 1997, Journal of Cognitive Neuroscience.

[45]  S. Zeki The distribution of wavelength and orientation selective cells in different areas of monkey visual cortex , 1983, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[46]  J. Lisman Bursts as a unit of neural information: making unreliable synapses reliable , 1997, Trends in Neurosciences.

[47]  T. Powell,et al.  The basic uniformity in structure of the neocortex. , 1980, Brain : a journal of neurology.

[48]  R. T. Bottle,et al.  References for reviews , 1989, J. Inf. Sci..

[49]  Marvin Minsky,et al.  Steps toward Artificial Intelligence , 1995, Proceedings of the IRE.

[50]  X. Wang Fast burst firing and short-term synaptic plasticity: A model of neocortical chattering neurons , 1999, Neuroscience.

[51]  J. A. Deutsch,et al.  On the category effect in visual search , 1977 .

[52]  D I Perrett,et al.  Visual Recognition Based on Temporal Cortex Cells: Viewer-Centred Processing of Pattern Configuration , 1998, Zeitschrift fur Naturforschung. C, Journal of biosciences.

[53]  J. Wolfe Visual search in continuous, naturalistic stimuli , 1994, Vision Research.

[54]  M. Wertheimer Laws of organization in perceptual forms. , 1938 .

[55]  Tj Sejnowski,et al.  Skeleton filters in the brain , 2014 .

[56]  Tomaso A. Poggio,et al.  Just One View: Invariances in Inferotemporal Cell Tuning , 1997, NIPS.

[57]  T. D. Albright,et al.  Transparency and coherence in human motion perception , 1990, Nature.

[58]  A. Peters,et al.  Neuronal organization in area 17 of cat visual cortex. , 1993, Cerebral cortex.

[59]  S. Hochstein,et al.  How Serial is Serial Processing in Vision? , 1989, Perception.

[60]  M. Posner,et al.  Does attention affect visual feature integration? , 1986, Journal of experimental psychology. Human perception and performance.

[61]  R. Tootell,et al.  Anatomical evidence for MT and additional cortical visual areas in humans. , 1995, Cerebral cortex.

[62]  B L McNaughton,et al.  Dynamics of the hippocampal ensemble code for space. , 1993, Science.

[63]  D. McCormick,et al.  Comparative electrophysiology of pyramidal and sparsely spiny stellate neurons of the neocortex. , 1985, Journal of neurophysiology.

[64]  E. Adelson,et al.  The analysis of moving visual patterns , 1985 .

[65]  K. Stratford,et al.  Synaptic transmission between individual pyramidal neurons of the rat visual cortex in vitro , 1991, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[66]  P. Milner A model for visual shape recognition. , 1974, Psychological review.

[67]  T J Sejnowski,et al.  A new view of hemineglect based on the response properties of parietal neurones. , 1997, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[68]  J. Lund,et al.  Widespread periodic intrinsic connections in the tree shrew visual cortex. , 1982, Science.

[69]  A. Sestokas,et al.  Response variability of X- and Y-cells in the dorsal lateral geniculate nucleus of the cat. , 1988, Journal of neurophysiology.

[70]  D. Ts'o,et al.  The organization of chromatic and spatial interactions in the primate striate cortex , 1988, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[71]  Christoph von der Malsburg,et al.  The What and Why of Binding The Modeler’s Perspective , 1999, Neuron.

[72]  John H. R. Maunsell,et al.  On the relationship between synaptic input and spike output jitter in individual neurons. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[73]  David I. Perrett,et al.  Neurophysiology of shape processing , 1993, Image Vis. Comput..

[74]  P. Cavanagh,et al.  FACIAL ORGANIZATION BLOCKS ACCESS TO LOW-LEVEL FEATURES: AN OBJECT INFERIORITY EFFECT , 1995 .

[75]  A Treisman,et al.  Feature analysis in early vision: evidence from search asymmetries. , 1988, Psychological review.

[76]  W Prinzmetal,et al.  Principles of feature integration in visual perception , 1981, Perception & psychophysics.

[77]  M N Shadlen,et al.  Motion perception: seeing and deciding. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[78]  Li I. Zhang,et al.  A critical window for cooperation and competition among developing retinotectal synapses , 1998, Nature.

[79]  Y. Miyashita,et al.  Neuronal tuning to learned complex forms in vision. , 1994, Neuroreport.

[80]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[81]  P. Schwindt,et al.  Amplification of synaptic current by persistent sodium conductance in apical dendrite of neocortical neurons. , 1995, Journal of neurophysiology.

[82]  K. Tanaka,et al.  Organization of cat visual cortex as investigated by cross-correlation technique. , 1981, Journal of neurophysiology.

[83]  M. Silverman,et al.  Spatial-frequency organization in primate striate cortex. , 1989, Proceedings of the National Academy of Sciences of the United States of America.

[84]  W. Singer Synchronization of cortical activity and its putative role in information processing and learning. , 1993, Annual review of physiology.

[85]  Leif H. Finkel,et al.  Perceptual grouping in striate cortical networks mediated by synchronization and desynchronization , 1999, Neurocomputing.

[86]  M. Tovée,et al.  Oscillatory activity is not evident in the primate temporal visual cortex with static stimuli , 1992, Neuroreport.

[87]  H. Nothdurft Feature analysis and the role of similarity in preattentive vision , 1992, Perception & psychophysics.

[88]  I. Biederman,et al.  Dynamic binding in a neural network for shape recognition. , 1992, Psychological review.

[89]  Christoph von der Malsburg,et al.  Pattern recognition by labeled graph matching , 1988, Neural Networks.

[90]  B Julesz,et al.  "Where" and "what" in vision. , 1985, Science.

[91]  O. Bertrand,et al.  Oscillatory gamma activity in humans and its role in object representation , 1999, Trends in Cognitive Sciences.

[92]  Steffen Schmalz,et al.  Combining Multiple Views and Temporal Associations for 3-D object Recognition , 1998, ECCV.

[93]  W. Newsome,et al.  The Variable Discharge of Cortical Neurons: Implications for Connectivity, Computation, and Information Coding , 1998, The Journal of Neuroscience.

[94]  Frank Rosenblatt,et al.  PRINCIPLES OF NEURODYNAMICS. PERCEPTRONS AND THE THEORY OF BRAIN MECHANISMS , 1963 .

[95]  E. Seidemann,et al.  Effect of spatial attention on the responses of area MT neurons. , 1999, Journal of neurophysiology.

[96]  W. Singer,et al.  The response of cat visual cortex to flicker stimuli of variable frequency , 1998, The European journal of neuroscience.

[97]  Wolfgang Konen,et al.  A fast dynamic link matching algorithm for invariant pattern recognition , 1994, Neural Networks.

[98]  John K. Tsotsos,et al.  Modeling Visual Attention via Selective Tuning , 1995, Artif. Intell..

[99]  D. Tank,et al.  Dendritic Integration in Mammalian Neurons, a Century after Cajal , 1996, Neuron.

[100]  R. Traub,et al.  A mechanism for generation of long-range synchronous fast oscillations in the cortex , 1996, Nature.

[101]  Marilyn Bohl,et al.  Information processing , 1971 .

[102]  John H. R. Maunsell,et al.  Visual response latencies in striate cortex of the macaque monkey. , 1992, Journal of neurophysiology.

[103]  J. B. Levitt,et al.  Relation between patterns of intrinsic lateral connectivity, ocular dominance, and cytochrome oxidase-reactive regions in macaque monkey striate cortex. , 1996, Cerebral cortex.

[104]  J. Storm Potassium currents in hippocampal pyramidal cells. , 1990, Progress in brain research.

[105]  K Tanaka,et al.  Neuronal mechanisms of object recognition. , 1993, Science.

[106]  A. Treisman The binding problem , 1996, Current Opinion in Neurobiology.

[107]  Victor A. F. Lamme,et al.  Contextual Modulation in Primary Visual Cortex , 1996, The Journal of Neuroscience.

[108]  W. Köhler The task of Gestalt psychology , 1969 .

[109]  S Marcelja,et al.  Mathematical description of the responses of simple cortical cells. , 1980, Journal of the Optical Society of America.

[110]  B. Sakmann,et al.  A new cellular mechanism for coupling inputs arriving at different cortical layers , 1999, Nature.

[111]  H. Sakata,et al.  Parietal neurons related to memory-guided hand manipulation. , 1996, Journal of neurophysiology.

[112]  A. Dale,et al.  New images from human visual cortex , 1996, Trends in Neurosciences.

[113]  J. Jonides,et al.  A conceptual category effect in visual search: O as letter or as digit , 1972 .

[114]  Ad Aertsen,et al.  Coding and Computation in the Cortex: Single-Neuron Activity and Cooperative Phenomena , 1992 .

[115]  Keiji Tanaka,et al.  Integration of direction signals of image motion in the superior temporal sulcus of the macaque monkey , 1986, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[116]  H. Jones,et al.  Visual cortical mechanisms detecting focal orientation discontinuities , 1995, Nature.

[117]  D. Hubel,et al.  Receptive fields and functional architecture of monkey striate cortex , 1968, The Journal of physiology.

[118]  Joel L. Davis,et al.  Large-Scale Neuronal Theories of the Brain , 1994 .

[119]  J. Wolfe,et al.  The Psychophysical Evidence for a Binding Problem in Human Vision , 1999, Neuron.

[120]  H. Nothdurft Faces and Facial Expressions do not Pop Out , 1993, Perception.

[121]  M. Stryker Is grandmother an oscillation? , 1989, Nature.

[122]  D H HUBEL,et al.  RECEPTIVE FIELDS AND FUNCTIONAL ARCHITECTURE IN TWO NONSTRIATE VISUAL AREAS (18 AND 19) OF THE CAT. , 1965, Journal of neurophysiology.

[123]  A. Treisman,et al.  Conjunction search revisited. , 1990, Journal of experimental psychology. Human perception and performance.

[124]  M. Stryker Cortical physiology. Is grandmother an oscillation? , 1989, Nature.

[125]  Bartlett W. Mel,et al.  Minimizing Binding Errors Using Learned Conjunctive Features , 2000, Neural Computation.

[126]  P. McLeod,et al.  Filtering by movement in visual search. , 1991, Journal of experimental psychology. Human perception and performance.

[127]  A. Treisman Visual Attention and the Perception of Features and Objects , 1994 .

[128]  S. Shipp,et al.  The functional logic of cortical connections , 1988, Nature.

[129]  D. Johnston,et al.  A Synaptically Controlled, Associative Signal for Hebbian Plasticity in Hippocampal Neurons , 1997, Science.

[130]  Hartmut Neven,et al.  The Bochum/USC Face Recognition System And How it Fared in the FERET Phase III Test , 1998 .

[131]  Catherine Tallon-Baudry,et al.  Induced γ-Band Activity during the Delay of a Visual Short-Term Memory Task in Humans , 1998, The Journal of Neuroscience.

[132]  K. H. Britten,et al.  Neuronal correlates of a perceptual decision , 1989, Nature.

[133]  D. Wilkin,et al.  Neuron , 2001, Brain Research.

[134]  G. Edelman,et al.  Consciousness and Complexity , 1998 .

[135]  V. Braitenberg Cell Assemblies in the Cerebral Cortex , 1978 .

[136]  M. Goodale,et al.  Chapter 28 Visual pathways to perception and action , 1993 .

[137]  John H. R. Maunsell,et al.  Shape selectivity in primate lateral intraparietal cortex , 1998, Nature.

[138]  Laurenz Wiskott,et al.  The role of topographical constraints in face recognition , 1999, Pattern Recognition Letters.

[139]  Marisa Carrasco,et al.  Attention improves or impairs visual performance by enhancing spatial resolution , 1998, Nature.

[140]  Martin I. Sereno,et al.  Cortical visual areas in mammals , 1991 .

[141]  W Singer,et al.  Visual feature integration and the temporal correlation hypothesis. , 1995, Annual review of neuroscience.

[142]  S. Zeki Functional specialisation in the visual cortex of the rhesus monkey , 1978, Nature.

[143]  William A. Phillips,et al.  On the acquisition of object concepts from sensory data , 1989 .

[144]  J. Szentágothai Synaptology of the Visual Cortex , 1973 .

[145]  Christoph von der Malsburg,et al.  The Correlation Theory of Brain Function , 1994 .

[146]  J. Bouyer,et al.  Fast somato-parietal rhythms during combined focal attention and immobility in baboon and squirrel monkey. , 1979, Electroencephalography and clinical neurophysiology.

[147]  E. Adelson Perceptual organization and the judgment of brightness. , 1993, Science.

[148]  Victor A. F. Lamme,et al.  Neuronal synchrony does not represent texture segregation , 1998, Nature.

[149]  A. Treisman Modularity and attention: Is the binding problem real? , 1995 .

[150]  K. Reinikainen,et al.  Selective attention enhances the auditory 40-Hz transient response in humans , 1993, Nature.

[151]  A. Parker,et al.  Sense and the single neuron: probing the physiology of perception. , 1998, Annual review of neuroscience.

[152]  Michael N. Shadlen,et al.  Noise, neural codes and cortical organization , 1994, Current Opinion in Neurobiology.

[153]  Charles M. Gray,et al.  Simulations of Intrinsically Bursting Neocortical Pyramidal Neurons , 1994, Neural Computation.

[154]  D. Purves,et al.  Number of `blobs' in the primary visual cortex of neonatal and adult monkeys , 1990, Proceedings of the National Academy of Sciences of the United States of America.

[155]  P. Lennie The physiological basis of variations in visual latency , 1981, Vision Research.

[156]  W. Singer,et al.  In search of common foundations for cortical computation , 1997, Behavioral and Brain Sciences.

[157]  Y. Miyashita,et al.  Neural organization for the long-term memory of paired associates , 1991, Nature.

[158]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[159]  B. Sakmann,et al.  Amplification of EPSPs by axosomatic sodium channels in neocortical pyramidal neurons , 1995, Neuron.