Thinking About The Brain

We all are fascinated by the phenomena of intelligent behavior, as generated both by our own brains and by the brains of other animals. As physicists we would like to understand if there are some general principles that govern the structure and dynamics of the neural circuits that underlie these phenomena. At the molecular level there is an extraordinary universality, but these mechanisms are surprisingly complex. This raises the question of how the brain selects from these diverse mechanisms and adapts to compute “the right thing” in each context. One approach is to ask what problems the brain really solves. There are several examples—from the ability of the visual system to count photons on a dark night to our gestalt recognition of statistical tendencies toward symmetry in random patterns—where the performance of the system in fact approaches some fundamental physical or statistical limits. This suggests that some sort of optimization principles may be at work, and there are examples where these principles have been formulated clearly and generated predictions which are confirmed in new experiments; a central theme in this work is the matching of the coding and computational strategies of the brain to the statistical structure of the world around us. Extension of these principles to the problem of learning leads us into interesting theoretical questions about how to measure the complexity of the data from which we learn and the complexity of the models that we use in learning, as well as opening some new opportunities for experiment. This combination of theoretical and experimental work gives us some new (if still speculative) perspectives on classical problems and controversies in cognition.

[1]  H. Wagner Flight Performance and Visual Control of Flight of the Free-Flying Housefly (Musca Domestica L.) I. Organization of the Flight Motor , 1986 .

[2]  William Bialek,et al.  Real-time performance of a movement-sensitive neuron in the blowfly visual system: coding and information transfer in short spike sequences , 1988, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[3]  G D Lewen,et al.  Reproducibility and Variability in Neural Spike Trains , 1997, Science.

[4]  W. Bialek,et al.  RELIABILITY AND STATISTICAL EFFICIENCY OF A BLOWFLY MOVEMENT-SENSITIVE NEURON , 1995 .

[5]  S. Laughlin A Simple Coding Procedure Enhances a Neuron's Information Capacity , 1981, Zeitschrift fur Naturforschung. Section C, Biosciences.

[6]  K. Hausen The Lobula-Complex of the Fly: Structure, Function and Significance in Visual Behaviour , 1984 .

[7]  H. V. Velden Over het aantal lichtquanta dat nodig is voor een lichtprikkel bij het menselijk oog , 1944 .

[8]  Pamela Reinagel,et al.  Decoding visual information from a population of retinal ganglion cells. , 1997, Journal of neurophysiology.

[9]  W Reichardt,et al.  Visual control of orientation behaviour in the fly: Part I. A quantitative analysis , 1976, Quarterly Reviews of Biophysics.

[10]  W. McCulloch,et al.  The limiting information capacity of a neuronal link , 1952 .

[11]  William Bialek,et al.  Coding and computation with neural spike trains , 1990 .

[12]  William Bialek,et al.  Entropy and Inference, Revisited , 2001, NIPS.

[13]  Sompolinsky,et al.  Statistical mechanics of learning from examples. , 1992, Physical review. A, Atomic, molecular, and optical physics.

[14]  J.,et al.  Dynamics of Rule Induction by Making Queries : Transition Between StrategiesIris , 1996 .

[15]  R A Mathies,et al.  Vibrationally coherent photochemistry in the femtosecond primary event of vision. , 1994, Science.

[16]  Paul R. Prucnal,et al.  Multiplication noise in the human visual system at threshold , 1982, Biological Cybernetics.

[17]  Naftali Tishby,et al.  Complexity through nonextensivity , 2001, physics/0103076.

[18]  Adrienne L. Fairhall,et al.  What Can a Single Neuron Compute? , 2000, NIPS.

[19]  M C Teich,et al.  Multiplication noise in the human visual system at threshold. 3. The role of non-Poisson quantum fluctuations. , 1982, Biological cybernetics.

[20]  D C Youvan,et al.  Direct observation of vibrational coherence in bacterial reaction centers using femtosecond absorption spectroscopy. , 1991, Proceedings of the National Academy of Sciences of the United States of America.

[21]  V. Periwal Reparametrization Invariant Statistical Inference and Gravity , 1997, hep-th/9703135.

[22]  D. Baylor,et al.  Responses of retinal rods to single photons. , 1979, The Journal of physiology.

[23]  Naftali Tishby,et al.  Document clustering using word clusters via the information bottleneck method , 2000, SIGIR '00.

[24]  F. Attneave Some informational aspects of visual perception. , 1954, Psychological review.

[25]  D. Baylor,et al.  Two components of electrical dark noise in toad retinal rod outer segments. , 1980, The Journal of physiology.

[26]  P. Grassberger Toward a quantitative theory of self-generated complexity , 1986 .

[27]  Naftali Tishby,et al.  Universality and Individuality in a Neural Code , 2000, NIPS.

[28]  R. H. S. Carpenter,et al.  Neural computation of log likelihood in control of saccadic eye movements , 1995, Nature.

[29]  E. Marder,et al.  Global Structure, Robustness, and Modulation of Neuronal Models , 2001, The Journal of Neuroscience.

[30]  Naftali Tishby,et al.  Predictability, Complexity, and Learning , 2000, Neural Computation.

[31]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[32]  H Barlow,et al.  Correspondence Noise and Signal Pooling in the Detection of Coherent Visual Motion , 1997, The Journal of Neuroscience.

[33]  R. Keynes The ionic channels in excitable membranes. , 1975, Ciba Foundation symposium.

[34]  E. Marder,et al.  A Model Neuron with Activity-Dependent Conductances Regulated by Multiple Calcium Sensors , 1998, The Journal of Neuroscience.

[35]  D. Baylor,et al.  Single-photon detection by rod cells of the retina , 1998 .

[36]  R N Aslin,et al.  Statistical Learning by 8-Month-Old Infants , 1996, Science.

[37]  Peter M. Vishton,et al.  Rule learning by seven-month-old infants. , 1999, Science.

[38]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1952, The Journal of physiology.

[39]  Martin Egelhaaf,et al.  Neural Mechanisms of Visual Course Control in Insects , 1989 .

[40]  William Bialek,et al.  Simple Models for the Dynamics of Biomolecules: How Far Can We Go? , 1987 .

[41]  R. Morris Foundations of cellular neurophysiology , 1996 .

[42]  Eve Marder,et al.  Network Stability from Activity-Dependent Regulation of Neuronal Conductances , 1999, Neural Computation.

[43]  Fred Rieke,et al.  Coding Efficiency and Information Rates in Sensory Neurons , 1993 .

[44]  William Bialek,et al.  Occam factors and model-independent Bayesian learning of continuous distributions , 2000, Physical review. E, Statistical, nonlinear, and soft matter physics.

[45]  W. Hilberg,et al.  Der bekannte Grenzwert der redundanzfreien Information in Texten - eine Fehlinterpretation der Shannonschen Experimente? , 1990 .

[46]  A.-C. Aho,et al.  Visual performance of the toad (Bufo bufo) at low light levels: retinal ganglion cell responses and prey-catching accuracy , 2004, Journal of Comparative Physiology A.

[47]  W. Bialek,et al.  Physical limits to sensation and perception. , 1987, Annual review of biophysics and biophysical chemistry.

[48]  Adrienne L. Fairhall,et al.  Computation in a Single Neuron: Hodgkin and Huxley Revisited , 2002, Neural Computation.

[49]  Hateren,et al.  Blowfly flight and optic flow. II. Head movements during flight , 1999, The Journal of experimental biology.

[50]  K. Donner,et al.  Low retinal noise in animals with low body temperature allows high visual sensitivity , 1988, Nature.

[51]  B. Sakmann,et al.  Single-channel currents recorded from membrane of denervated frog muscle fibres , 1976, Nature.

[52]  R. Hardie,et al.  Facets of Vision , 1989, Springer Berlin Heidelberg.

[53]  D. Baylor,et al.  The photocurrent, noise and spectral sensitivity of rods of the monkey Macaca fascicularis. , 1984, The Journal of physiology.

[54]  D. Baylor,et al.  Molecular origin of continuous dark noise in rod photoreceptors. , 1996, Biophysical journal.

[55]  S. Stevenson,et al.  Discrimination of jittered sonar echoes by the echolocating bat, Eptesicus fuscus: The shape of target images in echolocation , 1990, Journal of Comparative Physiology A.

[56]  William Bialek,et al.  Synergy in a Neural Code , 2000, Neural Computation.

[57]  H. Sullivan Ionic Channels of Excitable Membranes, 2nd Ed. , 1992, Neurology.

[58]  S. Laughlin,et al.  The rate of information transfer at graded-potential synapses , 1996, Nature.

[59]  Nicholas C. Spitzer,et al.  New dimensions of neuronal plasticity , 1999, Nature Neuroscience.

[60]  W. Bialek,et al.  Optimality and adaptation in motion estimation by the blowfly visual system , 1996, Proceedings of the IEEE 22nd Annual Northeast Bioengineering Conference.

[61]  Nick Chater,et al.  Distributional Information: A Powerful Cue for Acquiring Syntactic Categories , 1998, Cogn. Sci..

[62]  A. Hodgkin,et al.  The dual effect of membrane potential on sodium conductance in the giant axon of Loligo , 1952, The Journal of physiology.

[63]  J. H. Hateren,et al.  Theoretical predictions of spatiotemporal receptive fields of fly LMCs, and experimental validation , 1992, Journal of Comparative Physiology A.

[64]  Steven Abney,et al.  Statistical Methods and Linguistics , 2002 .

[65]  Vijay Balasubramanian,et al.  Statistical Inference, Occam's Razor, and Statistical Mechanics on the Space of Probability Distributions , 1996, Neural Computation.

[66]  R A Mathies,et al.  The first step in vision: femtosecond isomerization of rhodopsin. , 1991, Science.

[67]  Claude E. Shannon,et al.  Prediction and Entropy of Printed English , 1951 .

[68]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[69]  H B Barlow,et al.  The Ferrier lecture, 1980 , 1981, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[70]  D. Baylor,et al.  Origin of reproducibility in the responses of retinal rods to single photons. , 1998, Biophysical journal.

[71]  S. Leibler,et al.  Robustness in simple biochemical networks , 1997, Nature.

[72]  Naftali Tishby,et al.  Distributional Clustering of English Words , 1993, ACL.

[73]  M R Jones,et al.  Coherent nuclear dynamics at room temperature in bacterial reaction centers. , 1994, Proceedings of the National Academy of Sciences of the United States of America.

[74]  H. Barlow Critical limiting factors in the design of the eye and visual cortex , 1981 .

[75]  Adrienne L. Fairhall,et al.  Efficiency and ambiguity in an adaptive neural code , 2001, Nature.

[76]  A. Hodgkin,et al.  Currents carried by sodium and potassium ions through the membrane of the giant axon of Loligo , 1952, The Journal of physiology.

[77]  G D Lewen,et al.  Neural coding of naturalistic motion stimuli , 2001, Network.

[78]  T. Holy Analysis of Data from Continuous Probability Distributions , 1997, physics/9706015.

[79]  S Pinker,et al.  Out of the Minds of Babes , 1999, Science.

[80]  T. Aida Field Theoretical Analysis of On-line Learning of Probability Distributions , 1999, cond-mat/9911474.

[81]  Idan Segev,et al.  Ion Channel Stochasticity May Be Critical in Determining the Reliability and Precision of Spike Timing , 1998, Neural Computation.

[82]  H. Barlow The absolute efficiency of perceptual decisions. , 1980, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[83]  Elizabeth K. Johnson,et al.  Statistical learning of tone sequences by human infants and adults , 1999, Cognition.

[84]  Michael J. Berry,et al.  Adaptation of retinal processing to image contrast and spatial scale , 1997, Nature.

[85]  T. Collett,et al.  Chasing behaviour of houseflies (Fannia canicularis) , 1974, Journal of comparative physiology.

[86]  D. Ruderman,et al.  Statistics of cone responses to natural images: implications for visual coding , 1998 .

[87]  Naftali Tishby,et al.  The information bottleneck method , 2000, ArXiv.

[88]  Jerald D. Kralik,et al.  Real-time prediction of hand trajectory by ensembles of cortical neurons in primates , 2000, Nature.

[89]  R R Alfano,et al.  Fluorescence quantum yield of visual pigments: evidence for subpicosecond isomerization rates. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[90]  W. Bialek,et al.  Statistical mechanics and visual signal processing , 1994, cond-mat/9401072.

[91]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[92]  William Bialek,et al.  Spikes: Exploring the Neural Code , 1996 .

[93]  William Bialek,et al.  Entropy and Information in Neural Spike Trains , 1996, cond-mat/9603127.

[94]  Hateren,et al.  Blowfly flight and optic flow. I. Thorax kinematics and flight dynamics , 1999, The Journal of experimental biology.

[95]  W. Reichardt,et al.  Autocorrelation, a principle for the evaluation of sensory information by the central nervous system , 1961 .

[96]  E. Marder,et al.  Activity-dependent changes in the intrinsic properties of cultured neurons. , 1994, Science.

[97]  R. Fettiplace,et al.  The role of Ca2+‐activated K+ channel spliced variants in the tonotopic organization of the turtle cochlea , 1999, The Journal of physiology.

[98]  H. Barlow,et al.  Responses to single quanta of light in retinal ganglion cells of the cat. , 1971, Vision research.

[99]  E. Marder,et al.  Activity-dependent regulation of conductances in model neurons. , 1993, Science.

[100]  Lola L. Lopes,et al.  Distinguishing between random and nonrandom events. , 1987 .

[101]  A. Hodgkin,et al.  The components of membrane conductance in the giant axon of Loligo , 1952, The Journal of physiology.

[102]  C. Wehrhahn,et al.  Microsurgical lesion of horizontal cells changes optomotor yaw responses in the blowfly Calliphora erythrocephala , 1983, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[103]  S. Hecht,et al.  ENERGY, QUANTA, AND VISION , 1942, The Journal of general physiology.

[104]  Callan,et al.  Field Theories for Learning Probability Distributions. , 1996, Physical review letters.

[105]  Idan Segev,et al.  Information Capacity and Robustness of Stochastic Neuron Models , 1999, NIPS.

[106]  M R DeWeese,et al.  How to measure the information gained from one symbol. , 1999, Network.

[107]  M. Mauk,et al.  Learning-dependent timing of Pavlovian eyelid responses: differential conditioning using multiple interstimulus intervals. , 1992, Behavioral neuroscience.

[108]  R.R. de Ruyter Van Steveninck,et al.  Statistical adaptation and optimal estimation in movement computation by the blowfly visual system , 1994, Proceedings of IEEE International Conference on Systems, Man and Cybernetics.

[109]  William Bialek,et al.  Adaptive Rescaling Maximizes Information Transmission , 2000, Neuron.

[110]  Anirvan M. Sengupta,et al.  Engineering aspects of enzymatic signal transduction: photoreceptors in the retina. , 2000, Biophysical journal.

[111]  Ehud Zohary,et al.  Visual motion: linking neuronal activity to psychophysical performance , 1995 .

[112]  E. Marder,et al.  Activity-dependent modification of inhibitory synapses in models of rhythmic neural networks , 2001, Nature Neuroscience.

[113]  Alexander Borst,et al.  Real-Time Encoding of Motion: Answerable Questions and Questionable Answers from the Fly’s Visual System , 2000, physics/0004060.

[114]  N. Strausfeld Atlas of an Insect Brain , 1976, Springer Berlin Heidelberg.

[115]  H. Barlow Retinal noise and absolute threshold. , 1956, Journal of the Optical Society of America.

[116]  H. Barlow Intelligence, guesswork, language , 1983, Nature.

[117]  William Bialek,et al.  Statistics of Natural Images: Scaling in the Woods , 1993, NIPS.

[118]  Daniel P. Aalberts,et al.  Quantum coherent dynamics of molecules : A simple scenario for ultrafast photoisomerization , 2000 .

[119]  Niraj S. Desai,et al.  Plasticity in the intrinsic excitability of cortical pyramidal neurons , 1999, Nature Neuroscience.

[120]  Christof Koch,et al.  How voltage-dependent conductances can adapt to maximize the information encoded by neuronal firing rate , 1999, Nature Neuroscience.

[121]  William Bialek,et al.  Reading a Neural Code , 1991, NIPS.

[122]  Vos,et al.  Su-Schrieffer-Heeger model applied to chains of finite length. , 1996, Physical review. B, Condensed matter.