Toward an Integration of Deep Learning and Neuroscience

Neuroscience has focused on the detailed implementation of computation, studying neural codes, dynamics and circuits. In machine learning, however, artificial neural networks tend to eschew precisely designed codes, dynamics or circuits in favor of brute force optimization of a cost function, often using simple and relatively uniform initial architectures. Two recent developments have emerged within machine learning that create an opportunity to connect these seemingly divergent perspectives. First, structured architectures are used, including dedicated systems for attention, recursion and various forms of short- and long-term memory storage. Second, cost functions and training procedures have become more complex and are varied across layers and over time. Here we think about the brain in terms of these ideas. We hypothesize that (1) the brain optimizes cost functions, (2) the cost functions are diverse and differ across brain locations and over development, and (3) optimization operates within a pre-structured architecture matched to the computational problems posed by behavior. In support of these hypotheses, we argue that a range of implementations of credit assignment through multiple layers of neurons are compatible with our current knowledge of neural circuitry, and that the brain's specialized systems can be interpreted as enabling efficient optimization for specific problem classes. Such a heterogeneously optimized system, enabled by a series of interacting cost functions, serves to make learning data-efficient and precisely targeted to the needs of the organism. We suggest directions by which neuroscience could seek to refine and test these hypotheses.

[1]  HighWire Press Philosophical transactions of the Royal Society of London. B , 2022 .

[2]  R. Pearl Review: Nicolas Rashevsky, Mathematical Biophysics. Physicomathematical Foundations of Biology , 1939 .

[3]  Peter Elias,et al.  Predictive coding-I , 1955, IRE Trans. Inf. Theory.

[4]  G. A. Miller The magical number seven plus or minus two: some limits on our capacity for processing information. , 1956, Psychological review.

[5]  W. Pitts,et al.  What the Frog's Eye Tells the Frog's Brain , 1959, Proceedings of the IRE.

[6]  H. Lyman Mathematical biophysics: Physico-mathematical foundations of biology: N. Rashevsky. Third revised edition. Two vols. Dover, New York, 1960. Vol. I, xxvi + 488 pp. $2.50. Vol. II, xii + 462 pp. $2.50 , 1961 .

[7]  木村 憲二 巨視的成長の normative model について , 1961 .

[8]  N. Tinbergen Behavior and natural selection , 1965 .

[9]  S. Emlen Migratory Orientation in the Indigo Bunting, Passerina cyanea: Part I: Evidence for Use of Celestial Cues , 1967 .

[10]  D. Marr A theory of cerebellar cortex , 1969, The Journal of physiology.

[11]  E. Fetz Operant Conditioning of Cortical Unit Activity , 1969, Science.

[12]  Marvin Minsky,et al.  Perceptrons - an introduction to computational geometry , 1969 .

[13]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[14]  Marvin Minsky,et al.  Plain Talk about Neurodevelopmental Epistemology , 1977, IJCAI.

[15]  Sandy Lovie How the mind works , 1980, Nature.

[16]  HighWire Press The journal of neuroscience : the official journal of the Society for Neuroscience. , 1981 .

[17]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[18]  Paul J. Werbos,et al.  Applications of advances in nonlinear sensitivity analysis , 1982 .

[19]  L. M. Lieberman What If … , 1983, Journal of learning disabilities.

[20]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[21]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[22]  David Zipser,et al.  Feature discovery by competitive learning , 1986 .

[23]  B. Berger,et al.  Morphological evidence for a dopaminergic terminal field in the hippocampal formation of young and adult rat , 1985, Neuroscience.

[24]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[25]  D C Van Essen,et al.  Shifter circuits: a computational strategy for dynamic aspects of visual processing. , 1987, Proceedings of the National Academy of Sciences of the United States of America.

[26]  Pineda Generalization of back-propagation to recurrent neural networks. , 1987, Physical review letters.

[27]  A. Colino,et al.  Differential modulation of three separate K-conductances in hippocampal CA1 neurons by serotonin , 1987, Nature.

[28]  Geoffrey E. Hinton,et al.  Learning Representations by Recirculation , 1987, NIPS.

[29]  Richard A. Andersen,et al.  A back-propagation programmed network that simulates response properties of a subset of posterior parietal neurons , 1988, Nature.

[30]  D. G. Stork,et al.  Is backpropagation biologically plausible? , 1989, International 1989 Joint Conference on Neural Networks.

[31]  T. Poggio,et al.  A parallel algorithm for real-time computation of optical flow , 1989, Nature.

[32]  Francis Crick,et al.  The recent excitement about neural networks , 1989, Nature.

[33]  K. Miller,et al.  Ocular dominance column development: analysis and simulation. , 1989, Science.

[34]  Geoffrey E. Hinton Connectionist Learning Procedures , 1989, Artif. Intell..

[35]  Larry Austin Learning to Compose , 1989 .

[36]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[37]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990 .

[38]  J. Howard,et al.  Learning to Compose , 1990 .

[39]  John R. Anderson,et al.  Learning Artificial Grammars With Competitive Chunking , 1990 .

[40]  Peter Földiák,et al.  Learning Invariance from Transformation Sequences , 1991, Neural Comput..

[41]  G. Reeke The society of mind , 1991 .

[42]  Marvin Minsky,et al.  Society of Mind: A Response to Four Reviews , 1991, Artif. Intell..

[43]  Marvin Minsky,et al.  Logical Versus Analogical or Symbolic Versus Connectionist or Neat Versus Scruffy , 1991, AI Mag..

[44]  D. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[45]  Bartlett W. Mel The Clusteron: Toward a Simple Abstraction for a Complex Neuron , 1991, NIPS.

[46]  H. Karten,et al.  Multiple Origins of Neocortex: Contributions of the Dorsal Ventricular Ridge , 1991 .

[47]  D. Robinson Movement control: Implications of neural networks for how we think about brain function , 1992 .

[48]  Geoffrey E. Hinton,et al.  Self-organizing neural network that discovers surfaces in random-dot stereograms , 1992, Nature.


[50]  D. V. van Essen,et al.  A neurobiological model of visual attention and invariant pattern recognition based on dynamic routing of information , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[51]  Ronald J. Williams,et al.  Tight Performance Bounds on Greedy Policies Based on Imperfect Value Functions , 1993 .

[52]  J Miller Solving the problem. , 1993, Nursing times.

[53]  C. Koch,et al.  The highly irregular firing of cortical cells is inconsistent with temporal integration of random EPSPs , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[54]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[55]  Kenneth D. Miller,et al.  The Role of Constraints in Hebbian Learning , 1994, Neural Computation.

[56]  G. A. Miller The magical number seven, plus or minus two: some limits on our capacity for processing information. 1956. , 1994, Psychological review.

[57]  Gerald Tesauro,et al.  Temporal Difference Learning and TD-Gammon , 1995, J. Int. Comput. Games Assoc..

[58]  Tony A. Plate,et al.  Holographic reduced representations , 1995, IEEE Trans. Neural Networks.

[59]  G E Hinton,et al.  The "wake-sleep" algorithm for unsupervised neural networks. , 1995, Science.

[60]  M M Merzenich,et al.  Temporal information transformed into a spatial code by a neural network with realistic properties , 1995, Science.


[62]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[63]  P. McLeod,et al.  Do Fielders Know Where to Go to Catch the Ball or Only How to Get There , 1996 .

[64]  Randall C. O'Reilly,et al.  Biologically Plausible Error-Driven Learning Using Local Activation Differences: The Generalized Recirculation Algorithm , 1996, Neural Computation.

[65]  E. Kandel,et al.  Nitric Oxide Acts Directly in the Presynaptic Neuron to Produce Long-Term Potentiationin Cultured Hippocampal Neurons , 1996, Cell.

[66]  K. I. Blum,et al.  Functional significance of long-term potentiation for sequence learning and prediction. , 1996, Cerebral cortex.

[67]  Peter Dayan,et al.  Combining Probabilistic Population Codes , 1997, IJCAI.

[68]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[69]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[70]  H. Markram,et al.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs , 1997, Science.

[71]  Edward K. Vogel,et al.  The capacity of visual working memory for features and conjunctions , 1997, Nature.

[72]  D. Johnston,et al.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs , 1997 .

[73]  M. Hasselmo,et al.  Free recall and recognition in a network model of the hippocampus: simulating effects of scopolamine on human memory function , 1997, Behavioural Brain Research.

[74]  N. Kanwisher,et al.  The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception , 1997, The Journal of Neuroscience.

[75]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[76]  A. Graybiel The Basal Ganglia and Chunking of Action Repertoires , 1998, Neurobiology of Learning and Memory.

[77]  Daniel M. Wolpert,et al.  Signal-dependent noise determines motor planning , 1998, Nature.

[78]  A. Dickinson,et al.  Episodic-like memory during cache recovery by scrub jays , 1998, Nature.

[79]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .

[80]  H. Sebastian Seung,et al.  Continuous attractors and oculomotor control , 1998, Neural Networks.

[81]  Kenji Doya,et al.  What are the computations of the cerebellum, the basal ganglia and the cerebral cortex? , 1999, Neural Networks.

[82]  A. Gopnik The scientist in the crib , 1999 .

[83]  R. Desimone,et al.  The Role of Neural Mechanisms of Attention in Solving the Binding Problem , 1999, Neuron.

[84]  Xiaohui Xie,et al.  Spike-based Learning Rules and Stabilization of Persistent Neural Activity , 1999, NIPS.

[85]  Konrad P. Körding,et al.  A learning rule for dynamic recruitment and decorrelation , 2000, Neural Networks.

[86]  Terrence J. Sejnowski,et al.  Learning Overcomplete Representations , 2000, Neural Computation.

[87]  P. Dayan,et al.  A model of hippocampally dependent navigation, using the temporal difference learning rule , 2000, Hippocampus.

[88]  A. Meltzoff Born to Learn : What Infants Learn from Watching Us , 2000 .

[89]  Andrew Y. Ng,et al.  Algorithms for Inverse Reinforcement Learning , 2000, ICML.

[90]  G. Barreto,et al.  A SELF-ORGANIZING NEURAL NETWORK , 2000 .

[91]  D. Ferster,et al.  Neural mechanisms of orientation selectivity in the visual cortex. , 2000, Annual review of neuroscience.

[92]  G. Stuart,et al.  Backpropagation of Physiological Spike Trains in Neocortical Pyramidal Neurons: Implications for Temporal Coding in Dendrites , 2000, The Journal of Neuroscience.

[93]  A. Gopnik,et al.  The Scientist in the Crib: What Early Learning Tells Us About the Mind , 2000 .

[94]  Peter Redgrave,et al.  A computational model of action selection in the basal ganglia. I. A new functional anatomy , 2001, Biological Cybernetics.

[95]  Wulfram Gerstner,et al.  Intrinsic Stabilization of Output Rates by Spike-Based Hebbian Learning , 2001, Neural Computation.

[96]  R. Nicoll,et al.  Endogenous cannabinoids mediate retrograde signalling at hippocampal synapses , 2001, Nature.

[97]  G. Marcus The Algebraic Mind: Integrating Connectionism and Cognitive Science , 2001 .

[98]  William Bialek Thinking About The Brain , 2002 .

[99]  J. B. Levitt,et al.  Circuits for Local and Global Signal Integration in Primary Visual Cortex , 2002, The Journal of Neuroscience.

[100]  Danny Keogan,et al.  Distributed hierarchical processing , 2002, Photomask Japan.

[101]  Terrence J. Sejnowski,et al.  Slow Feature Analysis: Unsupervised Learning of Invariances , 2002, Neural Computation.

[102]  Michael I. Jordan,et al.  Optimal feedback control as a theory of motor coordination , 2002, Nature Neuroscience.

[103]  Lakhmi C. Jain,et al.  Self-Organizing Neural Networks , 2002 .

[104]  Janet D. Fodor,et al.  Understanding stimulus poverty arguments , 2002 .

[105]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[106]  Emanuel Todorov,et al.  Cosine Tuning Minimizes Motor Errors , 2002, Neural Computation.

[107]  H. Wellman,et al.  Infants' ability to connect gaze and emotional expression to intentional action , 2002, Cognition.

[108]  H. Seung,et al.  Robust persistent neural activity in a model integrator with multiple hysteretic dendrites per neuron. , 2003, Cerebral cortex.

[109]  Tai Sing Lee,et al.  Hierarchical Bayesian inference in the visual cortex. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[110]  H. Seung Learning in Spiking Neural Networks by Reinforcement of Stochastic Synaptic Transmission , 2003, Neuron.

[111]  Xiaohui Xie,et al.  Equivalence of Backpropagation and Contrastive Hebbian Learning in a Layered Network , 2003, Neural Computation.

[112]  Peter Dayan,et al.  Doubly Distributional Population Codes: Simultaneous Representation of Uncertainty and Multiplicity , 2003, Neural Computation.

[113]  Bruce D. McCandliss,et al.  The visual word form area: expertise for reading in the fusiform gyrus , 2003, Trends in Cognitive Sciences.

[114]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[115]  R. Douglas,et al.  Neuronal circuits of the neocortex. , 2004, Annual review of neuroscience.

[116]  P. König,et al.  How are complex cell properties adapted to the statistics of natural stimuli? , 2004, Journal of neurophysiology.

[117]  Andrzej Skowron,et al.  A parallel algorithm for real-time decision making: A rough set approach , 2004, Journal of Intelligent Information Systems.

[118]  D. Knill,et al.  The Bayesian brain: the role of uncertainty in neural coding and computation , 2004, Trends in Neurosciences.

[119]  Ronald J. Williams Simple statistical gradient-following algorithms for connectionist reinforcement learning , 2004, Machine Learning.

[120]  Redwood,et al.  What is the other 85 % of V 1 doing ? , 2004 .

[121]  R. Granger,et al.  Derivation and Analysis of Basic Computational Operations of Thalamocortical Circuits , 2004, Journal of Cognitive Neuroscience.

[122]  Kunihiko Fukushima,et al.  Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position , 1980, Biological Cybernetics.

[123]  Chris Eliasmith,et al.  Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems , 2004, IEEE Transactions on Neural Networks.

[124]  G. Marcus The Birth of the Mind: How a Tiny Number of Genes Creates The Complexities of Human Thought , 2004 .

[125]  Konrad P. Körding,et al.  Supervised and Unsupervised Learning with Two Sites of Synaptic Integration , 2004, Journal of Computational Neuroscience.

[126]  Tomaso Poggio,et al.  Generalization in vision and motor control , 2004, Nature.

[127]  Rajesh P. N. Rao Bayesian Computation in Recurrent Neural Circuits , 2004, Neural Computation.

[128]  Barak A. Pearlmutter,et al.  A normative model of attention: receptive field modulation , 2004, Neurocomputing.

[129]  David J. Field,et al.  What is the other 85% of V1 doing? , 2004 .

[130]  J. Hawkins,et al.  On Intelligence , 2004 .

[131]  Edward M. Callaway Feedforward, feedback and inhibitory connections in primate visual cortex , 2004, Neural Networks.

[132]  B. Scholl Can infants' object concepts be trained? , 2004, Trends in Cognitive Sciences.

[133]  Nello Cristianini,et al.  Supervised and Unsupervised Learning , 2004 .

[134]  L. Squire Memory systems of the brain: A brief history and current perspective , 2004, Neurobiology of Learning and Memory.

[135]  M. Meister,et al.  Dynamic predictive coding by the retina , 2005, Nature.

[136]  A. Lansner,et al.  The cortex as a central pattern generator , 2005, Nature Reviews Neuroscience.

[137]  E. Miller,et al.  Different time courses of learning-related activity in the prefrontal cortex and striatum , 2005, Nature.

[138]  C. Koch,et al.  What is the function of the claustrum? , 2005, Philosophical Transactions of the Royal Society B: Biological Sciences.

[139]  T. Simpatico The Birth of the Mind: How a Tiny Number of Genes Creates the Complexities of Human Thought , 2005 .

[140]  S. Sherman Thalamic relays and cortical functioning. , 2005, Progress in brain research.

[141]  Pieter R. Roelfsema,et al.  Attention-Gated Reinforcement Learning of Internal Representations for Classification , 2005, Neural Computation.

[142]  S. Grillner,et al.  Mechanisms for selection of basic motor programs – roles for the striatum and pallidum , 2005, Trends in Neurosciences.

[143]  Thomas Dean A Computational Model of the Cerebral Cortex , 2005, AAAI.

[144]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 2005, IEEE Transactions on Neural Networks.

[145]  Zoubin Ghahramani,et al.  A note on the evidence and Bayesian Occam's razor , 2005 .

[146]  H. Markram,et al.  The neocortical microcircuit as a tabula rasa. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[147]  Permalink Title Efficient Coding of Visual Scenes by Grouping and Segmentation : Theoretical Predictions and Biological Evidence , 2005 .

[148]  P. Dayan,et al.  Actions , Policies , Values , and the Basal Ganglia , 2005 .

[149]  Philipp Slusallek,et al.  Parallel & distributed processing , 2005, SIGGRAPH Courses.

[150]  Xiaohui Xie,et al.  Learning Curves for Stochastic Gradient Descent in Linear Feedforward Networks , 2003, Neural Computation.

[151]  Tai Sing Lee,et al.  Efficient Coding of Visual Scenes by Grouping and Segmentation , 2006 .

[152]  Ila R Fiete,et al.  Gradient learning in spiking neural networks by dynamic perturbation of conductances. , 2006, Physical review letters.

[153]  R. O’Reilly Biologically Based Computational Models of High-Level Cognition , 2006, Science.

[154]  Johannes J. Letzkus,et al.  Learning Rules for Spike Timing-Dependent Plasticity Depend on Dendritic Synapse Location , 2006, The Journal of Neuroscience.

[155]  Warren B. Powell,et al.  Handbook of Learning and Approximate Dynamic Programming , 2006, IEEE Transactions on Automatic Control.

[156]  Naftali Tishby,et al.  Efficient representation as a design principle for neural coding and computation , 2006, 2006 IEEE International Symposium on Information Theory.

[157]  M. Hasselmo The role of acetylcholine in learning and memory , 2006, Current Opinion in Neurobiology.

[158]  K. Obermayer,et al.  The Role of Feedback in Shaping the Extra-Classical Receptive Field of Cortical Neurons: A Recurrent Network Model , 2006, The Journal of Neuroscience.

[159]  Wei Ji Ma,et al.  Bayesian inference with probabilistic population codes , 2006, Nature Neuroscience.

[160]  E. Koechlin,et al.  Broca's Area and the Hierarchical Organization of Human Behavior , 2006, Neuron.

[161]  Michael J. Frank,et al.  Making Working Memory Work: A Computational Model of Learning in the Prefrontal Cortex and Basal Ganglia , 2006, Neural Computation.

[162]  W. Gerstner,et al.  Triplets of Spikes in a Model of Spike Timing-Dependent Plasticity , 2006, The Journal of Neuroscience.

[163]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[164]  E. Marder,et al.  Variability, compensation and homeostasis in neuron and network function , 2006, Nature Reviews Neuroscience.

[165]  Chen Shuo The Capacity of Visual Working Memory for Motion Direction of Objects , 2006 .

[166]  C. Eliasmith,et al.  Higher-Dimensional Neurons Explain the Tuning and Dynamics of Working Memory Cells , 2006, The Journal of Neuroscience.

[167]  David J. Field,et al.  What Is the Other 85 Percent of V1 Doing , 2006 .

[168]  P. König,et al.  A Model of the Ventral Visual System Based on Temporal Stability and Local Memory , 2006, PLoS biology.

[169]  Gerald M Edelman,et al.  A cerebellar model for predictive motor control tested in a brain-based device. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[170]  J. Tanji,et al.  Activity in the Lateral Prefrontal Cortex Reflects Multiple Steps of Future Events in Action Plans , 2006, Neuron.

[171]  Eugene M. Izhikevich,et al.  Polychronization: Computation with Spikes , 2006, Neural Computation.

[172]  P. J. Sjöström,et al.  A Cooperative Switch Determines the Sign of Synaptic Plasticity in Distal Dendrites of Neocortical Pyramidal Neurons , 2006, Neuron.

[173]  Thomas Serre,et al.  A Biologically Inspired System for Action Recognition , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[174]  E. Maguire Deconstructing episodic memory with , 2007 .

[175]  E. Fetz Volitional control of neural activity: implications for brain–computer interfaces , 2007, The Journal of physiology.

[176]  Thomas Serre,et al.  A feedforward architecture accounts for rapid categorization , 2007, Proceedings of the National Academy of Sciences.

[177]  John M. Ennis,et al.  A neurobiological theory of automaticity in perceptual categorization. , 2007, Psychological review.

[178]  Tom M. Mitchell,et al.  The Need for Biases in Learning Generalizations , 2007 .

[179]  Keith A. Bush An echo state model of non-markovian reinforcement learning , 2007 .

[180]  S. Sherman The thalamus is more than just a relay , 2007, Current Opinion in Neurobiology.

[181]  E. Izhikevich Solving the distal reward problem through linkage of STDP and dopamine signaling , 2007, BMC Neuroscience.

[182]  M. Hasselmo,et al.  Temporal Frequency of Subthreshold Oscillations Scales with Entorhinal Grid Cell Field Spacing , 2007, Science.

[183]  H. Seung,et al.  Model of birdsong learning based on gradient estimation by dynamic perturbation of neural conductances. , 2007, Journal of neurophysiology.

[184]  B. Dickson,et al.  Dscam diversity is essential for neuronal wiring and self-recognition , 2007, Nature.

[185]  S. Siegelbaum,et al.  A Role for Synaptic Inputs at Distal Dendrites: Instructive Signals for Hippocampal Long-Term Plasticity , 2007, Neuron.

[186]  Laurenz Wiskott,et al.  Slowness and Sparseness Lead to Place, Head-Direction, and Spatial-View Cells , 2007, PLoS Comput. Biol..

[187]  John R. Anderson How Can the Human Mind Occur in the Physical Universe , 2007 .

[188]  Karl J. Friston,et al.  Free-energy and the brain , 2007, Synthese.

[189]  Eduardo D. Sontag,et al.  Computational Aspects of Feedback in Neural Circuits , 2006, PLoS Comput. Biol..

[190]  D. Hassabis,et al.  Deconstructing episodic memory with construction , 2007, Trends in Cognitive Sciences.

[191]  Earl K Miller,et al.  The representation of multiple objects in prefrontal neuronal delay activity. , 2007, Cerebral cortex.

[192]  M. Wilson,et al.  Coordinated memory replay in the visual cortex and hippocampus during sleep , 2007, Nature Neuroscience.

[193]  J. Hamlin,et al.  Social evaluation by preverbal infants , 2007, Nature.

[194]  K. Körding Decision Theory: What "Should" the Nervous System Do? , 2007, Science.

[195]  Richard G. Baraniuk,et al.  Sparse Coding via Thresholding and Local Competition in Neural Circuits , 2008, Neural Computation.

[196]  Brian A. Wandell,et al.  Population receptive field estimates in human visual cortex , 2008, NeuroImage.

[197]  Chris Eliasmith,et al.  Solving the Problem of Negative Synaptic Weights in Cortical Models , 2008, Neural Computation.

[198]  K. Harris Stability of the fittest: organizing learning through retroaxonal signals , 2008, Trends in Neurosciences.

[199]  Tomaso A. Poggio,et al.  A Canonical Neural Circuit for Cortical Nonlinear Operations , 2008, Neural Computation.

[200]  Nikolaus Kriegeskorte,et al.  Representational Similarity Analysis – Connecting the Branches of Systems Neuroscience , 2008, Frontiers in systems neuroscience.

[201]  D. Hassabis,et al.  Tracking the Emergence of Conceptual Knowledge during Human Decision Making , 2009, Neuron.

[202]  John J Hopfield Neurodynamics of mental exploration , 2009, Proceedings of the National Academy of Sciences.

[203]  G. Perea,et al.  Tripartite synapses: astrocytes process and control synaptic information , 2009, Trends in Neurosciences.

[204]  Emanuel Todorov,et al.  Efficient computation of optimal actions , 2009, Proceedings of the National Academy of Sciences.

[205]  M. Botvinick,et al.  Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective , 2009, Cognition.

[206]  Ana Calabrese,et al.  Ocular dominance column , 2009, Scholarpedia.

[207]  Robert F. Hadley The Problem of Rapid Variable Creation , 2009, Neural Computation.

[208]  L. F. Abbott,et al.  Generating Coherent Patterns of Activity from Chaotic Neural Networks , 2009, Neuron.

[209]  Willem H. Zuidema,et al.  Simple rules can explain discrimination of putative recursive syntactic structures by a songbird species , 2009, Proceedings of the National Academy of Sciences.

[210]  Jason Weston,et al.  Curriculum learning , 2009, ICML '09.

[211]  Demis Hassabis,et al.  The construction system of the brain , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[212]  Markus Siegel,et al.  Phase-dependent neuronal coding of objects in short-term memory , 2009, Proceedings of the National Academy of Sciences.

[213]  Suzanna Becker,et al.  Computational cognitive neuroscience , 2009, Brain Research.

[214]  Pascal Vincent,et al.  The Difficulty of Training Deep Architectures and the Effect of Unsupervised Pre-Training , 2009, AISTATS.

[215]  Dileep George,et al.  Towards a Mathematical Theory of Cortical Micro-circuits , 2009, PLoS Comput. Biol..

[216]  N. Kanwisher,et al.  The Cognitive and Neural Development of Face Recognition in Humans , 2009 .

[217]  Dean V Buonomano,et al.  Embedding Multiple Trajectories in Simulated Recurrent Neural Networks in a Self-Organizing Manner , 2009, The Journal of Neuroscience.

[218]  Hossein Mobahi,et al.  Deep learning from temporal coherence in video , 2009, ICML '09.

[219]  Markus Werning,et al.  Compositionality and Biologically Plausible Models , 2009 .

[220]  Karl J. Friston The free-energy principle: a unified brain theory? , 2010, Nature Reviews Neuroscience.

[221]  M. Hasselmo,et al.  Coupled Noisy Spiking Neurons as Velocity-Controlled Oscillators in a Model of Grid Cell Spatial Firing , 2010, The Journal of Neuroscience.

[222]  V. Ramachandran Mirror neurons and imitation learning as the driving force behind "the great leap forward" in human evolution , 2010 .

[223]  M. Botvinick,et al.  Abstract Structural Representations of Goal-Directed Behavior , 2010, Psychological science.

[224]  Jürgen Schmidhuber,et al.  Formal Theory of Creativity, Fun, and Intrinsic Motivation (1990–2010) , 2010, IEEE Transactions on Autonomous Mental Development.

[225]  M. Nardini,et al.  Fusion of visual cues is not mandatory in children , 2010, Proceedings of the National Academy of Sciences.

[226]  E. Miller,et al.  Task-Dependent Changes in Short-Term Memory in the Prefrontal Cortex , 2010, The Journal of Neuroscience.

[227]  Wulfram Gerstner,et al.  Spike-timing dependent plasticity , 2010, Scholarpedia.

[228]  Takeo Watanabe,et al.  Perceptual learning rules based on reinforcers and attention , 2010, Trends in Cognitive Sciences.

[229]  Benjamin O. Turner,et al.  Cortical and basal ganglia contributions to habit learning and automaticity , 2010, Trends in Cognitive Sciences.

[230]  David J. Jilk,et al.  The Leabra architecture: Specialization without modularity , 2010 .

[231]  C. Lebiere,et al.  Conditional routing of information to the cortex: a model of the basal ganglia's role in cognitive coordination. , 2010, Psychological review.

[232]  H. Coslett,et al.  Interval timing disruptions in subjects with cerebellar lesions , 2010, Neuropsychologia.

[233]  T. Poggio,et al.  What and where: A Bayesian inference theory of attention , 2010, Vision Research.

[234]  Chris Eliasmith,et al.  A Spiking Neuron Model of Serial-Order Recall , 2010 .

[235]  Wen-Liang L Zhou,et al.  The decade of the dendritic NMDA spike , 2010, Journal of neuroscience research.

[236]  Earl K. Miller,et al.  Shifting the Spotlight of Attention: Evidence for Discrete Computations in Cognition , 2010, Front. Hum. Neurosci..

[237]  Terrence C. Stewart,et al.  Symbolic Reasoning in Spiking Neurons: A Model of the Cortex/Basal Ganglia/Thalamus Loop , 2010 .

[238]  György Buzsáki Neural Syntax: Cell Assemblies, Synapsembles, and Readers , 2010, Neuron.

[239]  M. Desmurget,et al.  Basal ganglia contributions to motor control: a vigorous tutor , 2010, Current Opinion in Neurobiology.

[240]  Risto Miikkulainen,et al.  Hebbian Learning , 2010, Encyclopedia of Machine Learning.

[241]  Richard Hans Robert Hahnloser,et al.  Spike-Time-Dependent Plasticity and Heterosynaptic Competition Organize Networks to Produce Long Scale-Free Sequences of Neural Activity , 2010, Neuron.

[242]  Wulfram Gerstner,et al.  Voltage and Spike Timing Interact in STDP – A Unified Model , 2010, Front. Syn. Neurosci..

[243]  H. Eichenbaum,et al.  Hippocampal “Time Cells” Bridge the Gap in Memory for Discontiguous Events , 2011, Neuron.

[244]  Ilya Sutskever,et al.  Learning Recurrent Neural Networks with Hessian-Free Optimization , 2011, ICML.

[245]  Geoffrey E. Hinton,et al.  Transforming Auto-Encoders , 2011, ICANN.

[246]  M. Fee,et al.  Two Distinct Modes of Forebrain Circuit Dynamics Underlie Temporal Patterning in the Vocalizations of Young Songbirds , 2011, The Journal of Neuroscience.

[247]  Yoshua Bengio,et al.  Shallow vs. Deep Sum-Product Networks , 2011, NIPS.

[248]  Soren Solari,et al.  Cognitive Consilience: Primate Non-Primary Neuroanatomical Circuits Underlying Cognition , 2011, Front. Neuroanat..

[249]  E. Molnár Long-term potentiation in cultured hippocampal neurons. , 2011, Seminars in cell & developmental biology.

[250]  G. Shepherd The Microcircuit Concept Applied to Cortical Evolution: from Three-Layer to Six-Layer Cortex , 2011, Front. Neuroanat..

[251]  Simon J. Thorpe,et al.  Low-Level Cues and Ultra-Fast Face Detection , 2011, Front. Psychology.

[252]  H. Hirase,et al.  Astrocyte Calcium Signaling Transforms Cholinergic Modulation to Cortical Plasticity In Vivo , 2011, The Journal of Neuroscience.

[253]  Yi Sun,et al.  Planning to Be Surprised: Optimal Bayesian Exploration in Dynamic Environments , 2011, AGI.

[254]  I. Biederman,et al.  Neural encoding of relative position. , 2011, Journal of experimental psychology. Human perception and performance.

[255]  T DeWolf,et al.  The neural optimal control hierarchy for motor control. , 2011, Journal of neural engineering.

[256]  J. Bizley The Neural Bases of Multisensory Processes , 2011 .

[257]  J. Nassi,et al.  Segregation of feedforward and feedback projections in mouse visual cortex , 2011, The Journal of comparative neurology.

[258]  Rajesh P. N. Rao,et al.  Predictive coding. , 2011, Wiley interdisciplinary reviews. Cognitive science.

[259]  Scott V Taylor,et al.  Does the cost function of human motor control depend on the internal metabolic state? , 2011, BMC Neuroscience.

[260]  Karl F. Stock,et al.  A COMPUTATIONAL MODEL , 2011 .

[261]  Patrick Henry Winston The Strong Story Hypothesis and the Directed Perception Hypothesis , 2011, AAAI Fall Symposium: Advances in Cognitive Systems.

[262]  Geoffrey E. Hinton,et al.  Generating Text with Recurrent Neural Networks , 2011, ICML.

[263]  Wolfgang Maass,et al.  Neural Dynamics as Sampling: A Model for Stochastic Computation in Recurrent Networks of Spiking Neurons , 2011, PLoS Comput. Biol..

[264]  Wolfgang Maass,et al.  Branch-Specific Plasticity Enables Self-Organization of Nonlinear Computation in Single Neurons , 2011, The Journal of Neuroscience.

[265]  Chris Eliasmith,et al.  Normalization for probabilistic inference with neurons , 2011, Biological Cybernetics.

[266]  Niko Wilbert,et al.  Slow feature analysis , 2011, Scholarpedia.

[267]  Kenneth J. Hayworth Dynamically Partitionable Autoassociative Networks as a Solution to the Neural Binding Problem , 2012, Front. Comput. Neurosci..

[268]  Cori Bargmann Beyond the connectome: How neuromodulators shape neural circuits , 2012, BioEssays : news and reviews in molecular, cellular and developmental biology.

[269]  Trevor Bekolay,et al.  A Large-Scale Model of the Functioning Brain , 2012, Science.

[270]  Per B. Sederberg,et al.  The Successor Representation and Temporal Context , 2012, Neural Computation.

[271]  James J. DiCarlo,et al.  How Does the Brain Solve Visual Object Recognition? , 2012, Neuron.

[272]  S. G. Andino,et al.  Coding of saliency by ensemble bursting in the amygdala of primates , 2012, Front. Behav. Neurosci..

[273]  M. Frank,et al.  Mechanisms of hierarchical reinforcement learning in corticostriatal circuits 1: computational analysis. , 2012, Cerebral cortex.

[274]  P. Barone,et al.  Cortical and Thalamic Pathways for Multisensory and Sensorimotor Interplay , 2012 .

[275]  A. Gopnik,et al.  Learning about causes from people: observational causal learning in 24-month-old infants. , 2012, Developmental psychology.

[276]  Peter Dayan,et al.  Efficient Bayes-Adaptive Reinforcement Learning using Sample-Based Search , 2012, NIPS.

[277]  N. Rashevsky Mathematical Biophysics: Physicomathematical Foundations Of Biology , 2012 .

[278]  Joel Z. Leibo,et al.  Learning and disrupting invariance in visual recognition with a temporal association rule , 2012, Front. Comput. Neurosci..

[279]  P. Dayan Twenty-Five Lessons from Computational Neuromodulation , 2012, Neuron.

[280]  G. Turrigiano Homeostatic synaptic plasticity: local and global mechanisms for stabilizing neuronal function. , 2012, Cold Spring Harbor perspectives in biology.

[281]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[282]  Steven L Franconeri,et al.  A simple proximity heuristic allows tracking of multiple objects through occlusion , 2012, Attention, perception & psychophysics.

[283]  Zoran Popovic,et al.  Discovery of complex behaviors through contact-invariant optimization , 2012, ACM Trans. Graph..

[284]  Shimon Ullman,et al.  From simple innate biases to complex visual concepts , 2012, Proceedings of the National Academy of Sciences.

[285]  J. DiCarlo,et al.  Neuronal Learning of Invariant Object Representation in the Ventral Visual Stream Is Not Dependent on Reward , 2012, The Journal of Neuroscience.

[286]  R. Berwick,et al.  A Bird’s Eye View of Human Language Evolution , 2012, Front. Evol. Neurosci..

[287]  Geoffrey E. Hinton,et al.  Deep Mixtures of Factor Analysers , 2012, ICML.

[288]  Linda B. Smith,et al.  Joint Attention without Gaze Following: Human Infants and Their Parents Coordinate Visual Attention to Objects through Eye-Hand Coordination , 2013, PloS one.

[289]  E. Marder,et al.  From the connectome to brain function , 2013, Nature Methods.

[290]  L. Abbott,et al.  Random Convergence of Olfactory Inputs in the Drosophila Mushroom Body , 2013, Nature.

[291]  André van Schaik,et al.  Learning the pseudoinverse solution to network weights , 2013, Neural Networks.

[292]  Robert C. Berwick,et al.  The Emergence of Hierarchical Structure in Human Language , 2013, Front. Psychol..

[293]  R. Friedrich,et al.  Neural Circuits: Random Design of a Higher-Order Olfactory Projection , 2013, Current Biology.

[294]  Wolfgang Maass,et al.  Bayesian Computation Emerges in Generic Cortical Microcircuits through Spike-Timing-Dependent Plasticity , 2013, PLoS Comput. Biol..

[295]  A. Meltzoff,et al.  11 Developmental Perspectives on Action Science: Lessons from Infant Imitation and Cognitive Neuroscience , 2013 .

[296]  Jessica B. Hamrick,et al.  Simulation as an engine of physical scene understanding , 2013, Proceedings of the National Academy of Sciences.

[297]  Roi Livni,et al.  An Algorithm for Training Polynomial Networks , 2013 .

[298]  Geoffrey E. Hinton,et al.  Tensor Analyzers , 2013, ICML.

[299]  Trenton E. Kriete,et al.  Strategic Cognitive Sequencing: A Computational Cognitive Neuroscience Approach , 2013, Comput. Intell. Neurosci..

[300]  J. Maunsell,et al.  Insights into cortical mechanisms of behavior from microstimulation experiments , 2013, Progress in Neurobiology.

[301]  Richard Hans Robert Hahnloser,et al.  A Hebbian learning rule gives rise to mirror neurons and links them to control theoretic inverse models , 2013, Front. Neural Circuits.

[302]  Edmund T. Rolls The mechanisms for pattern completion and pattern separation in the hippocampus , 2013, Front. Syst. Neurosci..

[303]  Jonathan D. Cohen,et al.  Indirection and symbol-like processing in the prefrontal cortex and basal ganglia , 2013, Proceedings of the National Academy of Sciences.

[304]  C. Pennartz,et al.  A unified selection signal for attention and reward in primary visual cortex , 2013, Proceedings of the National Academy of Sciences.

[305]  Stephen Grossberg Adaptive Resonance Theory: How a brain learns to consciously attend, learn, and recognize a changing world , 2013, Neural Networks.

[306]  Marc'Aurelio Ranzato,et al.  Building high-level features using large scale unsupervised learning , 2011, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[307]  G. Buzsáki,et al.  Memory, navigation and theta rhythm in the hippocampal-entorhinal system , 2013, Nature Neuroscience.

[308]  Xiao-Jing Wang The Prefrontal Cortex as a Quintessential “Cognitive-Type” Neural Circuit , 2013 .

[309]  Gilles Wainrib,et al.  A Biological Gradient Descent for Prediction Through a Combination of STDP and Homeostatic Plasticity , 2013, Neural Computation.

[310]  Chris Eliasmith,et al.  How to Build a Brain: A Neural Architecture for Biological Cognition , 2013 .

[311]  M. Hasselmo,et al.  Hippocampal “Time Cells”: Time versus Path Integration , 2013, Neuron.

[312]  M. Bear,et al.  A Cholinergic Mechanism for Reward Timing within Primary Visual Cortex , 2013, Neuron.

[313]  Yann LeCun,et al.  Pedestrian Detection with Unsupervised Multi-stage Feature Learning , 2012, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[314]  John J. Hopfield,et al.  Rapid, parallel path planning by propagating wavefronts of spiking neural activity , 2013, Front. Comput. Neurosci..

[315]  Geoffrey E. Hinton,et al.  On the importance of initialization and momentum in deep learning , 2013, ICML.

[316]  Trevor Bekolay,et al.  Simultaneous unsupervised and supervised learning of cognitive functions in biologically plausible spiking neural networks , 2013, CogSci.

[317]  Joseph G. Makin,et al.  Learning Multisensory Integration and Coordinate Transformation via Density Estimation , 2013, PLoS Comput. Biol..

[318]  M. Larkum A cellular mechanism for cortical associations: an organizing principle for the cerebral cortex , 2013, Trends in Neurosciences.

[319]  Xiao-Jing Wang,et al.  The importance of mixed selectivity in complex cognitive tasks , 2013, Nature.

[320]  José Carlos Príncipe,et al.  Deep Predictive Coding Networks , 2013, ICLR.

[321]  Y. Dan,et al.  Representation of interval timing by temporally scalable firing patterns in rat prefrontal cortex , 2013, Proceedings of the National Academy of Sciences.

[322]  N. Strausfeld,et al.  Deep Homology of Arthropod Central Complex and Vertebrate Basal Ganglia , 2013, Science.

[323]  G. Rubin,et al.  Mushroom body output neurons encode valence and guide memory-based action selection in Drosophila , 2014, eLife.

[324]  Ari Weinstein,et al.  Model-based hierarchical reinforcement learning and human action control , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[325]  P. Roelfsema,et al.  Alpha and gamma oscillations characterize feedback and feedforward processing in monkey visual cortex , 2014, Proceedings of the National Academy of Sciences.

[326]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[327]  Seth A. Herd,et al.  Goal-Driven Cognition in the Brain: A Computational Framework , 2014, 1404.7591.

[328]  Scott T Grafton,et al.  Multifaceted aspects of chunking enable robust algorithms. , 2014, Journal of neurophysiology.

[329]  Thomas Dean,et al.  The atoms of neural computation , 2014, Science.

[330]  Terrence J. Sejnowski,et al.  Prospective Optimization , 2014, Proceedings of the IEEE.

[331]  E. Spelke,et al.  Preverbal infants identify emotional reactions that are incongruent with goal outcomes , 2014, Cognition.

[332]  Harri Valpola,et al.  From neural PCA to deep unsupervised learning , 2014, ArXiv.

[333]  Adam H. Marblestone,et al.  Designing Tools for Assumption-Proof Brain Mapping , 2014, Neuron.

[334]  Terrence C. Stewart,et al.  A Unifying Mechanistic Model of Selective Attention in Spiking Neurons , 2014, PLoS Comput. Biol..

[335]  M. Fee,et al.  A role for descending auditory cortical projections in songbird vocal learning , 2014, eLife.

[336]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[337]  David Kappel,et al.  STDP Installs in Winner-Take-All Circuits an Online Approximation to Hidden Markov Model Learning , 2014, PLoS Comput. Biol..

[338]  Ben Goertzel How might the brain represent complex symbolic knowledge? , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[339]  L. F. Abbott,et al.  Hierarchical Control Using Networks Trained with Higher-Level Forward Models , 2014, Neural Computation.

[340]  M. Larkum,et al.  NMDA spikes enhance action potential generation during sensory input , 2014, Nature Neuroscience.

[341]  David Sussillo Neural circuits as computational dynamical systems , 2014, Current Opinion in Neurobiology.

[342]  Samuel Gershman,et al.  Time representation in reinforcement learning models of the basal ganglia , 2014, Front. Comput. Neurosci..

[343]  David Balduzzi,et al.  Cortical prediction markets , 2014, AAMAS.

[344]  Samuel Gershman,et al.  Design Principles of the Hippocampal Cognitive Map , 2014, NIPS.

[345]  Vikash K. Mansinghka,et al.  Building fast Bayesian computing machines out of intentionally stochastic, digital parts , 2014, ArXiv.

[346]  Joshua B. Tenenbaum,et al.  When Computer Vision Gazes at Cognition , 2014, ArXiv.

[347]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2014, ICLR.

[348]  Randall C. O'Reilly,et al.  Learning Through Time in the Thalamocortical Loops , 2014 .

[349]  Kenneth D. Harris,et al.  The Neural Marketplace: I. General Formalism and Linear Theory , 2014 .

[350]  Yoshua Bengio,et al.  How Auto-Encoders Could Provide Credit Assignment in Deep Networks via Target Propagation , 2014, ArXiv.

[351]  Program Size and Temperature in Self-Assembly , 2010, Algorithmica.

[352]  Alex Graves,et al.  Recurrent Models of Visual Attention , 2014, NIPS.

[353]  Adam H. Marblestone,et al.  Frequently Asked Questions for: The Atoms of Neural Computation , 2014, bioRxiv.

[354]  W. Senn,et al.  Learning by the Dendritic Prediction of Somatic Spiking , 2014, Neuron.

[355]  E. Miller,et al.  Goal-direction and top-down control , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[356]  Surya Ganguli,et al.  Exact solutions to the nonlinear dynamics of learning in deep linear neural networks , 2014, ICLR.

[357]  Daniel Cownden,et al.  Random feedback weights support learning in deep neural networks , 2014, ArXiv.

[358]  Christian Balkenius,et al.  The principles of goal-directed decision-making: from neural mechanisms to computation and robotics , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[359]  Sridhar Mahadevan,et al.  Modeling Context in Cognition Using Variational Inequalities , 2014, AAAI Fall Symposia.

[360]  Joan Bruna,et al.  Training Convolutional Networks with Noisy Labels , 2014, ICLR 2014.

[361]  R. Ivry,et al.  Generalized Role for the Cerebellum in Encoding Internal Models: Evidence from Semantic Processing , 2014, The Journal of Neuroscience.

[362]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[363]  Thomas L. Dean,et al.  Neural Networks and Neuroscience-Inspired Computer Vision , 2014, Current Biology.

[364]  L. Luo,et al.  Genetic Control of Wiring Specificity in the Fly Olfactory System , 2014, Genetics.

[365]  Ivan Laptev,et al.  Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[366]  Morten H. Christiansen,et al.  A computational model , 2014 .

[367]  Alex Graves,et al.  Neural Turing Machines , 2014, ArXiv.

[368]  G. Hesslow,et al.  Memory trace and timing mechanism localized to cerebellar Purkinje cells , 2014, Proceedings of the National Academy of Sciences.

[369]  Rich Caruana,et al.  Do Deep Nets Really Need to be Deep? , 2013, NIPS.

[370]  W. Gerstner,et al.  Optimal Control of Transient Dynamics in Balanced Networks Supports Generation of Complex Movements , 2014, Neuron.

[371]  X. Tolsa The capacity γ , 2014 .

[372]  Nikolaus Kriegeskorte,et al.  Deep Supervised, but Not Unsupervised, Models May Explain IT Cortical Representation , 2014, PLoS Comput. Biol..

[373]  Ned T. Sahin,et al.  Dynamic circuit motifs underlying rhythmic gain control, gating and integration , 2014, Nature Neuroscience.

[374]  Richard Hans Robert Hahnloser,et al.  Evidence for a causal inverse model in an avian cortico-basal ganglia circuit , 2014, Proceedings of the National Academy of Sciences.

[375]  G. Rubin,et al.  The neuronal architecture of the mushroom body provides a logic for associative learning , 2014, eLife.

[376]  Wolfgang Maass,et al.  Emergence of complex computational structures from chaotic neural networks through reward-modulated Hebbian learning. , 2014, Cerebral cortex.

[377]  Byron M. Yu,et al.  Neural constraints on learning , 2014, Nature.

[378]  B. Hangya,et al.  Central Cholinergic Neurons Are Rapidly Recruited by Reinforcement Feedback , 2015, Cell.

[379]  Dan Klein,et al.  Deep Compositional Question Answering with Neural Module Networks , 2015, ArXiv.

[380]  D. Hassabis,et al.  Hippocampal place cells construct reward related sequences through unexplored space , 2015, eLife.

[381]  Steven M Frankland,et al.  An architecture for encoding sentence meaning in left mid-superior temporal cortex , 2015, Proceedings of the National Academy of Sciences.

[382]  Joseph E O'Doherty,et al.  A learning–based approach to artificial sensory feedback leads to optimal integration , 2015, Nature Neuroscience.

[383]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[384]  L. Wilbrecht,et al.  Between the primate and ‘reptilian’ brain: Rodent models demonstrate the role of corticostriatal circuits in decision making , 2015, Neuroscience.

[385]  Yoshua Bengio,et al.  Towards Biologically Plausible Deep Learning , 2015, ArXiv.

[386]  Yaniv Ziv,et al.  Hippocampal ensemble dynamics timestamp events in long-term memory , 2015, eLife.

[387]  Pieter R. Roelfsema,et al.  Reinforcement Learning of Linking and Tracing Contours in Recurrent Neural Networks , 2015, PLoS Comput. Biol..

[388]  C. Curtis,et al.  Multiple component networks support working memory in prefrontal cortex , 2015, Proceedings of the National Academy of Sciences.

[389]  Qi Zhao,et al.  Foveation-based Mechanisms Alleviate Adversarial Examples , 2015, ArXiv.

[390]  Sophie Denève,et al.  Enforcing balance allows local supervised learning in spiking recurrent networks , 2015, NIPS.

[391]  S. Linnarsson,et al.  Cell types in the mouse cortex and hippocampus revealed by single-cell RNA-seq , 2015, Science.

[392]  J. Fournier,et al.  Looking for the roots of cortical sensory computation in three-layered cortices , 2015, Current Opinion in Neurobiology.

[393]  Guillaume Bouchard,et al.  Accelerating Stochastic Gradient Descent via Online Learning to Sample , 2015, ArXiv.

[394]  Oriol Vinyals,et al.  Qualitatively characterizing neural network optimization problems , 2014, ICLR.

[395]  Wojciech Zaremba,et al.  Reinforcement Learning Neural Turing Machines - Revised , 2015 .

[396]  Florent Meyniel,et al.  The Neural Representation of Sequences: From Transition Probabilities to Algebraic Patterns and Linguistic Trees , 2015, Neuron.

[397]  Guillaume Charpiat,et al.  Training recurrent networks online without backtracking , 2015, ArXiv.

[398]  R. Brownstone,et al.  Spinal circuits for motor learning , 2015, Current Opinion in Neurobiology.

[399]  Alexander S. Ecker,et al.  Principles of connectivity among morphologically defined cell types in adult neocortex , 2015, Science.

[400]  Peter Dayan,et al.  Interplay of approximate planning strategies , 2015, Proceedings of the National Academy of Sciences.

[401]  Joshua B. Tenenbaum,et al.  Deep Convolutional Inverse Graphics Network , 2015, NIPS.

[402]  M. Shapiro,et al.  A Map for Social Navigation in the Human Brain , 2015, Neuron.

[403]  Joel Z. Leibo,et al.  Approximate Hubel-Wiesel Modules and the Data Structures of Neural Computation , 2015, ArXiv.

[404]  M. Hasselmo If I had a million neurons: Potential tests of cortico-hippocampal theories. , 2015, Progress in brain research.

[405]  Jason Weston,et al.  Memory Networks , 2014, ICLR.

[406]  Yoshua Bengio,et al.  STDP as presynaptic activity times rate of change of postsynaptic activity , 2015 .

[407]  Sanjeev Arora,et al.  Why are deep nets reversible: A simple theory, with implications for training , 2015, ArXiv.

[408]  Joel Z. Leibo,et al.  The Invariance Hypothesis Implies Domain-Specific Regions in Visual Cortex , 2015, PLoS Comput. Biol..

[409]  Joel L. Voss,et al.  Covert rapid action-memory simulation (CRAMS): A hypothesis of hippocampal–prefrontal interactions for adaptive behavior , 2015, Neurobiology of Learning and Memory.

[410]  A. Nobre,et al.  Time in Cortical Circuits , 2015, The Journal of Neuroscience.

[411]  Dmitri B. Chklovskii,et al.  Optimization theory of Hebbian/anti-Hebbian networks for PCA and whitening , 2015, 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[412]  Joshua B. Tenenbaum,et al.  Human-level concept learning through probabilistic program induction , 2015, Science.

[413]  Jianing Yu,et al.  Low-noise encoding of active touch by layer 4 in the somatosensory cortex , 2015, eLife.

[414]  James G. King,et al.  Reconstruction and Simulation of Neocortical Microcircuitry , 2015, Cell.

[415]  Tapani Raiko,et al.  Semi-supervised Learning with Ladder Networks , 2015, NIPS.

[416]  Joachim M. Buhmann,et al.  Kickback Cuts Backprop's Red-Tape: Biologically Plausible Credit Assignment in Neural Networks , 2014, AAAI.

[417]  H. Kennedy,et al.  Visual Areas Exert Feedforward and Feedback Influences through Distinct Frequency Channels , 2014, Neuron.

[418]  Raphael Cohn,et al.  Coordinated and Compartmentalized Neuromodulation Shapes Sensory Processing in Drosophila , 2015, Cell.

[419]  Christof Koch,et al.  Physiology of Layer 5 Pyramidal Neurons in Mouse Primary Visual Cortex: Coincidence Detection through Bursting , 2015, PLoS Comput. Biol..

[420]  Alex Graves,et al.  DRAW: A Recurrent Neural Network For Image Generation , 2015, ICML.

[421]  Joscha Bach,et al.  Request Confirmation Networks for Neuro-Symbolic Script Execution , 2015, CoCo@NIPS.

[422]  Shakir Mohamed,et al.  Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning , 2015, NIPS.

[423]  Richard G. Baraniuk,et al.  A Probabilistic Theory of Deep Learning , 2015, ArXiv.

[424]  Joscha Bach Modeling Motivation in MicroPsi 2 , 2015, AGI.

[425]  Gabriel Kreiman,et al.  Unsupervised Learning of Visual Structure using Predictive Generative Networks , 2015, ArXiv.

[426]  Martin A. Riedmiller,et al.  Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images , 2015, NIPS.

[427]  Yoshua Bengio,et al.  Variance Reduction in SGD by Distributed Importance Sampling , 2015, ArXiv.

[428]  Alan L. Yuille,et al.  Learning Deep Structured Models , 2014, ICML.

[429]  Pierre Baldi,et al.  The Ebb and Flow of Deep Learning: a Theory of Local Learning , 2015, ArXiv.

[430]  Yoshua Bengio,et al.  FitNets: Hints for Thin Deep Nets , 2014, ICLR.

[431]  M. Hasselmo,et al.  Current questions on space and time encoding , 2015, Hippocampus.

[432]  Matthew T. Kaufman,et al.  A neural network that finds a naturalistic solution for the production of muscle activity , 2015, Nature Neuroscience.

[433]  Amanda L. Loshbaugh,et al.  Labelling and optical erasure of synaptic memory traces in the motor cortex , 2015, Nature.

[434]  Yasser Roudi,et al.  Learning with hidden variables , 2015, Current Opinion in Neurobiology.

[435]  Tomas Mikolov,et al.  Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets , 2015, NIPS.

[436]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[437]  Ryan P. Adams,et al.  Gradient-based Hyperparameter Optimization through Reversible Learning , 2015, ICML.

[438]  Paul Miller,et al.  Spiking neuron network Helmholtz machine , 2015, Front. Comput. Neurosci..

[439]  Alan L. Yuille,et al.  Semantic part segmentation using compositional model combining shape and appearance , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[440]  F. Tong,et al.  Neural mechanisms of object-based attention. , 2015, Cerebral cortex.

[441]  Yoshua Bengio Early Inference in Energy-Based Models Approximates Back-Propagation , 2015, ArXiv.

[442]  Marcel A. J. van Gerven,et al.  Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream , 2015 .

[443]  Pieter R. Roelfsema,et al.  How Attention Can Create Synaptic Tags for the Learning of Working Memories in Sequential Tasks , 2015, PLoS Comput. Biol..

[444]  Wojciech Zaremba,et al.  Reinforcement Learning Neural Turing Machines , 2015, ArXiv.

[445]  Scott P. Johnson,et al.  Perception of Object Persistence: The Origins of Object Permanence in Infancy , 2015 .

[446]  David Kappel,et al.  Network Plasticity as Bayesian Inference , 2015, PLoS Comput. Biol..

[447]  Andrew Zisserman,et al.  Spatial Transformer Networks , 2015, NIPS.

[448]  W. Gan,et al.  Branch-specific dendritic Ca2+ spikes cause persistent synaptic plasticity , 2015, Nature.

[449]  Mark M. Churchland,et al.  Using Firing-Rate Dynamics to Train Recurrent Networks of Spiking Model Neurons , 2016, 1601.07620.

[450]  Sergey Levine,et al.  Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization , 2016, ICML.

[451]  Joshua B. Tenenbaum,et al.  The Naïve Utility Calculus: Computational Principles Underlying Commonsense Psychology , 2016, Trends in Cognitive Sciences.

[452]  Daan Wierstra,et al.  One-Shot Generalization in Deep Generative Models , 2016, ICML.

[453]  Balaraman Ravindran,et al.  Hierarchical Reinforcement Learning using Spatio-Temporal Abstractions and Deep Neural Networks , 2016, ArXiv.

[454]  Valerie A. Carr,et al.  Prospective representation of navigational goals in the human hippocampus , 2016, Science.

[455]  Chris Eliasmith,et al.  Function approximation in inhibitory networks , 2016, Neural Networks.

[456]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[457]  T. Bugnyar,et al.  Cognition without Cortex , 2016, Trends in Cognitive Sciences.

[458]  L. F. Abbott,et al.  Building functional networks of spiking model neurons , 2016, Nature Neuroscience.

[459]  Joshua B. Tenenbaum,et al.  Understanding Visual Concepts with Continuation Learning , 2016, ArXiv.

[460]  Daniel R. Berger,et al.  The Fuzzy Logic of Network Connectivity in Mouse Visual Thalamus , 2016, Cell.

[461]  Geoffrey E. Hinton,et al.  Attend, Infer, Repeat: Fast Scene Understanding with Generative Models , 2016, NIPS.

[462]  Paolo Favaro,et al.  Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles , 2016, ECCV.

[463]  Yoshua Bengio,et al.  Feedforward Initialization for Fast Inference of Deep Generative Networks is biologically plausible , 2016, ArXiv.

[464]  Wulfram Gerstner,et al.  Nonlinear Hebbian Learning as a Unifying Principle in Receptive Field Formation , 2016, PLoS Comput. Biol..

[465]  Richard Granger,et al.  The grammar of mammalian brain capacity , 2016, Theor. Comput. Sci..

[466]  Wulfram Gerstner,et al.  Does computational neuroscience need new synaptic learning paradigms? , 2016, Current Opinion in Behavioral Sciences.

[467]  Koray Kavukcuoglu,et al.  Pixel Recurrent Neural Networks , 2016, ICML.

[468]  Stefano Ermon,et al.  Generative Adversarial Imitation Learning , 2016, NIPS.

[469]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[470]  Csaba Petre,et al.  Unsupervised Learning from Continuous Video in a Scalable Predictive Recurrent Network , 2016, ArXiv.

[471]  Yoshua Bengio,et al.  Knowledge Matters: Importance of Prior Information for Optimization , 2013, J. Mach. Learn. Res..

[472]  Timothy E. J. Behrens,et al.  Organizing conceptual knowledge in humans with a gridlike code , 2016, Science.

[473]  R. Baillargeon,et al.  Psychological Reasoning in Infancy. , 2016, Annual review of psychology.

[474]  Charles Blundell,et al.  Early Visual Concept Learning with Unsupervised Deep Learning , 2016, ArXiv.

[475]  Joshua B. Tenenbaum,et al.  Building machines that learn and think like people , 2016, Behavioral and Brain Sciences.

[476]  Dan Klein,et al.  Learning to Compose Neural Networks for Question Answering , 2016, HLT-NAACL.

[477]  Yoshua Bengio,et al.  Towards a Biologically Plausible Backprop , 2016, ArXiv.

[478]  Linda B. Smith,et al.  From faces to hands: Changing visual input in the first two years , 2016, Cognition.

[479]  Johanni Brea,et al.  Prospective Coding by Spiking Neurons , 2016, PLoS Comput. Biol..

[480]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[481]  Pieter Abbeel,et al.  Value Iteration Networks , 2016, NIPS.

[482]  James L. McClelland,et al.  What Learning Systems do Intelligent Agents Need? Complementary Learning Systems Theory Updated , 2016, Trends in Cognitive Sciences.

[483]  M. Nardini,et al.  Risky visuomotor choices during rapid reaching in childhood , 2015, Developmental science.

[484]  J. Schiller,et al.  A Novel Form of Local Plasticity in Tuft Dendrites of Neocortical Somatosensory Layer 5 Pyramidal Neurons , 2016, Neuron.

[485]  Eric Jonas,et al.  Could a Neuroscientist Understand a Microprocessor? , 2016 .

[486]  Mark S. Cembrowski,et al.  Structured Dendritic Inhibition Supports Branch-Selective Integration in CA1 Pyramidal Cells , 2016, Neuron.

[487]  Christopher D. Harvey,et al.  Recurrent Network Models of Sequence Generation and Memory , 2016, Neuron.

[488]  Hilbert J. Kappen,et al.  Learning Universal Computations with Spikes , 2016, PLoS Comput. Biol..

[489]  J. DiCarlo,et al.  Using goal-driven deep learning models to understand sensory cortex , 2016, Nature Neuroscience.

[490]  Joseph G. Makin,et al.  Recurrent Exponential-Family Harmoniums without Backprop-Through-Time , 2016, ArXiv.

[491]  Brent Komer,et al.  A unified theoretical approach for biological cognition and learning , 2016, Current Opinion in Behavioral Sciences.

[492]  D. Wolpert,et al.  Computations underlying sensorimotor learning , 2016, Current Opinion in Neurobiology.

[493]  Marcin Andrychowicz,et al.  Neural Random Access Machines , 2015, ERCIM News.

[494]  Tomaso A. Poggio,et al.  Bridging the Gaps Between Residual Learning, Recurrent Neural Networks and Visual Cortex , 2016, ArXiv.

[495]  Subutai Ahmad,et al.  Why Neurons Have Thousands of Synapses, a Theory of Sequence Memory in Neocortex , 2016, Front. Neural Circuits.

[496]  Wei Ji Ma,et al.  The Inevitability of Probability: Probabilistic Inference in Generic Neural Networks Trained with Non-Probabilistic Feedback , 2016 .

[497]  Sergey Levine,et al.  End-to-End Training of Deep Visuomotor Policies , 2015, J. Mach. Learn. Res..

[498]  Joel Z. Leibo,et al.  Model-Free Episodic Control , 2016, ArXiv.

[499]  Daan Wierstra,et al.  One-shot Learning with Memory-Augmented Neural Networks , 2016, ArXiv.

[500]  Walter Senn,et al.  Somato-dendritic Synaptic Plasticity and Error-backpropagation in Active Dendrites , 2016, PLoS Comput. Biol..

[501]  Quoc V. Le,et al.  Neural Programmer: Inducing Latent Programs with Gradient Descent , 2015, ICLR.

[502]  Sarah Noami Lewis The neural marketplace , 2016 .

[503]  Wolfgang Maass Searching for principles of brain computation , 2016, Current Opinion in Behavioral Sciences.

[504]  Alcino J. Silva,et al.  A shared neural ensemble links distinct contextual memories encoded close in time , 2016, Nature.

[505]  Joshua B. Tenenbaum,et al.  Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation , 2016, NIPS.

[506]  Lorenzo Rosasco,et al.  Unsupervised learning of invariant representations , 2016, Theor. Comput. Sci..

[507]  Alex Graves,et al.  Associative Long Short-Term Memory , 2016, ICML.

[508]  P. Jonas,et al.  Symmetric spike timing-dependent plasticity at CA3–CA3 synapses optimizes storage and recall in autoassociative networks , 2016, Nature communications.

[509]  J. Dudman,et al.  Opponent and bidirectional control of movement velocity in the basal ganglia , 2016, Nature.

[510]  James J DiCarlo,et al.  Eight open questions in the computational modeling of higher sensory cortex , 2016, Current Opinion in Neurobiology.

[511]  D. Hassabis,et al.  Neural Mechanisms of Hierarchical Planning in a Virtual Subway Network , 2016, Neuron.

[512]  Joshua B Tenenbaum,et al.  Toward the neural implementation of structure learning , 2016, Current Opinion in Neurobiology.

[513]  Joel Z. Leibo,et al.  How Important Is Weight Symmetry in Backpropagation? , 2015, AAAI.

[514]  Peter Ford Dominey,et al.  Reservoir Computing Properties of Neural Dynamics in Prefrontal Cortex , 2016, PLoS Comput. Biol..

[515]  Serge J. Belongie,et al.  Residual Networks are Exponential Ensembles of Relatively Shallow Networks , 2016, ArXiv.

[516]  Yuwei Cui,et al.  Continuous Online Sequence Learning with an Unsupervised Neural Network Model , 2016, Neural Computation.

[517]  Tomaso Poggio,et al.  Spatio-temporal convolutional neural networks explain human neural representations of action recognition , 2016 .

[518]  Richard Socher,et al.  Dynamic Memory Networks for Visual and Textual Question Answering , 2016, ICML.

[519]  Samuel J. Gershman,et al.  Complex Probabilistic Inference , 2017 .

[520]  Alex Graves,et al.  Decoupled Neural Interfaces using Synthetic Gradients , 2016, ICML.

[521]  Peter Stone,et al.  Reinforcement Learning , 2010, Encyclopedia of Machine Learning and Data Mining.

[522]  Gabriel Kreiman,et al.  Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning , 2016, ICLR.

[523]  Konrad Paul Körding,et al.  Could a Neuroscientist Understand a Microprocessor? , 2017, PLoS Comput. Biol..

[524]  Ashley B. Lyons,et al.  Inferring Social Disposition by Sound and Surface Appearance in Infancy , 2017 .

[525]  Ivo D. Dinov Deep Learning, Neural Networks , 2018 .

[526]  Bianca Dumitrascu,et al.  End-to-end training of deep probabilistic CCA for joint modeling of paired biomedical observations , 2018 .