In Neural Computation

Lecture Notes for the MSc/DTC module. The brain is a complex computing machine which has evolved to give the ttest output to a given input. Neural computation has as goal to describe the function of the nervous system in mathematical and computational terms. By analysing or simulating the resulting equations, one can better understand its function, research how changes in parameters would eect the function, and try to mimic the nervous system in hardware or software implementations. Neural Computation is a bit like physics, that has been successful in describing numerous physical phenomena. However, approaches developed in those elds not always work for neural computation, because: 1. Physical systems are best studied in reduced, simplied circumstances, but the nervous system is hard to study in isolation. Neurons require a narrow range of operating conditions (temperature, oxygen, presence of other neurons, ion concentrations, ...) under which they work as they should. These conditions are hard to reproduce outside the body. Secondly, the neurons form a highly interconnected network. The function of the nervous systems depends on this connectivity and interaction, by trying to isolate the components, you are likely to alter the function. 2. It is not clear how much detail one needs to describe the computations in the brain. In these lectures we shall see various description levels. 3. Neural signals and neural connectivity are hard to measure, especially, if disturbance and damage to the nervous system is to be kept minimal. Perhaps Neural Computation has more in common with trying to gure out how a complicated machine, such as a computer or car works. Knowledge of the basic physics helps, but is not sucient. Luckily there are factors which perhaps make understanding the brain easier than understanding an arbitrary complicated machine: 1. There is a high degree of conservation across species. This means that animal studies can be used to gain information about the human brain. Furthermore, study of, say, the visual system might help to understand the auditory system. 2. The nervous system is able to develop by combining on one hand a only limited amount of genetic information and, on the other hand, the input it receives. Therefore it might be possible to nd the organising principles and develop a brain from there. This would be easier than guring out the complete 'wiring diagram'. 3. The nervous system is exible and robust, neurons die everyday. This stands …

[1]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[2]  H. Evans.  The Study of Instinct , 1952 .

[3]  H. Barlow.  Temporal and spatial summation in human vision at different background intensities , 1958, The Journal of physiology.

[4]  A. Huxley.  ION MOVEMENTS DURING NERVE ACTIVITY , 1959, Annals of the New York Academy of Sciences.

[5]  W. Pitts,et al.  What the Frog's Eye Tells the Frog's Brain , 1959, Proceedings of the IRE.

[6]  B. Mandelbrot,et al.  RANDOM WALK MODELS FOR THE SPIKE ACTIVITY OF A SINGLE NEURON. , 1964, Biophysical journal.

[7]  H. Barlow,et al.  The mechanism of directionally selective units in rabbit's retina. , 1965, The Journal of physiology.

[8]  Grace Jordison.  Molecular Biology of the Gene , 1965, The Yale Journal of Biology and Medicine.

[9]  A. Luria.  Objectives of Clinicopsychological Investigation of Higher Cortical Functions in the Presence of Local Brain Lesions , 1966 .

[10]  John von Neumann,et al.  Theory Of Self Reproducing Automata , 1967 .

[11]  Herbert A. Simon,et al.  The Sciences of the Artificial , 1970 .

[12]  C F Stevens,et al.  Inferences about membrane properties from electrical noise measurements. , 1972, Biophysical journal.

[13]  C. R. Fourtner,et al.  NERVOUS CONTROL OF WALKING IN THE COCKROACH , 1973 .

[14]  T. Bliss,et al.  Long‐lasting potentiation of synaptic transmission in the dentate area of the anaesthetized rabbit following stimulation of the perforant path , 1973, The Journal of physiology.

[15]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[16]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[17]  H. L. Bryant,et al.  Spike initiation by transmembrane current: a white‐noise analysis. , 1976, The Journal of physiology.

[18]  T. Sejnowski.  Statistical constraints on synaptic plasticity. , 1977, Journal of theoretical biology.

[19]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[20]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[21]  E. Bienenstock,et al.  Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[22]  G. Buchsbaum,et al.  Trichromacy, opponent colours coding and optimum colour information transmission in the retina , 1983, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[23]  B. Hille.  Ionic channels of excitable membranes , 2001 .

[24]  R. von der Heydt,et al.  Illusory contours and cortical neuron responses. , 1984, Science.

[25]  C. Langton.  Self-reproduction in cellular automata , 1984 .

[26]  Stuart A. Kauffman.  Boolean systems, adaptive automata, evolution , 1986 .

[27]  M. Alexander,et al.  Principles of Neural Science , 1981 .

[28]  J. Hopfield,et al.  Computing with neural circuits: a model. , 1986, Science.

[29]  John H. Holland,et al.  Escaping brittleness: the possibilities of general-purpose learning algorithms applied to parallel rule-based systems , 1995 .

[30]  Stephen Wolfram,et al.  Theory and Applications of Cellular Automata , 1986 .

[31]  S. Kauffman,et al.  Adaptive automata based on Darwinian selection , 1986 .

[32]  J. Moreno.  The Man Who Mistook His Wife for a Hat and Other Clinical Tales , 1986 .

[33]  J. P. Jones,et al.  The two-dimensional spatial structure of simple receptive fields in cat striate cortex. , 1987, Journal of neurophysiology.

[34]  Tommaso Toffoli,et al.  Cellular Automata Machines , 1987, Complex Syst..

[35]  Conrad Schneiker,et al.  Nanotechnology with Feynman Machines: Scanning Tunneling Engineering and Artificial Life , 1987, ALIFE.

[36]  W. Bialek,et al.  Physical limits to sensation and perception. , 1987, Annual review of biophysics and biophysical chemistry.

[37]  Dan Hammerstrom,et al.  An interconnect structure for wafer scale neurocomputers , 1988, Neural Networks.

[38]  The Mind Machine , 1988 .

[39]  Henry C. Tuckwell,et al.  Introduction to theoretical neurobiology , 1988 .

[40]  D Stauffer.  Percolation thresholds in square-lattice Kauffman model. , 1988, Journal of theoretical biology.

[41]  Rajarshi Das,et al.  A Study of Control Parameters Affecting Online Performance of Genetic Algorithms for Function Optimization , 1989, ICGA.

[42]  Barak A. Pearlmutter.  Learning State Space Trajectories in Recurrent Neural Networks , 1989, Neural Computation.

[43]  Ronald J. Williams,et al.  Experimental Analysis of the Real-time Recurrent Learning Algorithm , 1989 .

[44]  Daniel L. Stein,et al.  Lectures In The Sciences Of Complexity , 1989 .

[45]  P. Goldman-Rakic,et al.  Mnemonic coding of visual space in the monkey's dorsolateral prefrontal cortex. , 1989, Journal of neurophysiology.

[46]  Bernard Manderick,et al.  Fine-Grained Parallel Genetic Algorithms , 1989, ICGA.

[47]  Alan Watt,et al.  Fundamentals of three-dimensional computer graphics , 1989 .

[48]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[49]  K. H. Britten,et al.  Neuronal correlates of a perceptual decision , 1989, Nature.

[50]  John R. Koza,et al.  Hierarchical Genetic Algorithms Operating on Populations of Computer Programs , 1989, IJCAI.

[51]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[52]  G. Edelman,et al.  Signal and sense : local and global order in perceptual maps , 1990 .

[53]  Rodney A. Brooks,et al.  Learning to Coordinate Behaviors , 1990, AAAI.

[54]  In Schoenauer,et al.  Parallel Problem Solving from Nature , 1990, Lecture Notes in Computer Science.

[55]  Stuart A. Kauffman,et al.  Requirements for evolvability in complex systems: orderly dynamics and frozen components , 1990 .

[56]  M. Ono.  [Scanning tunneling microscope]. , 1990, Tanpakushitsu kakusan koso. Protein, nucleic acid, enzyme.

[57]  John R. Koza,et al.  Genetic programming: a paradigm for genetically breeding populations of computer programs to solve problems , 1990 .

[58]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1990 .

[59]  Kenneth J. Hintz,et al.  Evolving a neural network , 1990, Proceedings. 5th IEEE International Symposium on Intelligent Control 1990.

[60]  G. Reeke,et al.  Selectionist models of perceptual and motor systems and implications for functionalist theories of brain function , 1990 .

[61]  H. Wallinga,et al.  SEAS: a simulated evolution approach for analog circuit synthesis , 1991, Proceedings of the IEEE 1991 Custom Integrated Circuits Conference.

[62]  William A. Goddard,et al.  Theoretical studies of a hydrogen abstraction tool for nanotechnology , 1991 .

[63]  Thomas S. Ray,et al.  Is It Alive or Is It GA? , 1991, ICGA.

[64]  Peter Földiák,et al.  Learning Invariance from Transformation Sequences , 1991, Neural Comput..

[65]  Thomas S. Ray,et al.  An Approach to the Synthesis of Life , 1991 .

[66]  P König,et al.  Direct physiological evidence for scene segmentation by temporal coding. , 1991, Proceedings of the National Academy of Sciences of the United States of America.

[67]  Stuart A. Kauffman,et al.  The origins of order , 1993 .

[68]  G. Reeke.  Marvin Minsky, The Society of Mind , 1991, Artif. Intell..

[69]  P. Holmes,et al.  1989 Lectures in Complex Systems , 1991 .

[70]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[71]  Y. Miyashita,et al.  Neural organization for the long-term memory of paired associates , 1991, Nature.

[72]  Bernard Manderick,et al.  A Massively Parallel Genetic Algorithm: Implementation and First Analysis , 1991, ICGA.

[73]  P. Todd,et al.  Exploring Adaptive Agency I: Theory and Methods for Simulating the Evolution of Learning , 1991 .

[74]  S. Kauffman,et al.  Antichaos and adaptation. , 1991, Scientific American.

[75]  Moshe Abeles,et al.  Corticonics: Neural Circuits of Cerebral Cortex , 1991 .

[76]  Alan S. Perelson,et al.  Molecular evolution on rugged landscapes : proteins, RNA and the immune system : the proceedings of the Workshop on Applied Molecular Evolution and the Maturation of the Immune Response, held March, 1989 in Santa Fe, New Mexico , 1991 .

[77]  Charles E. Taylor,et al.  Artificial Life II , 1991 .

[78]  Bernard Manderick,et al.  The Genetic Algorithm and the Structure of the Fitness Landscape , 1991, ICGA.