Inferring neural activity before plasticity as a foundation for learning beyond backpropagation

[1]  R. Bogacz,et al.  Predictive coding networks for temporal prediction , 2024, bioRxiv.

[2]  W. Senn,et al.  A neuronal least-action principle for real-time learning in cortical circuits , 2024, bioRxiv.

[3]  Benjamin F. Grewe,et al.  Minimizing Control for Credit Assignment with Strong Feedback , 2022, ICML.

[4]  Thomas Lukasiewicz,et al.  Learning on Arbitrary Graph Topologies via Predictive Coding , 2022, NeurIPS.

[5]  Walter Senn,et al.  Latent Equilibrium: A unified learning theory for arbitrarily fast computation with arbitrarily slow neurons , 2021, NeurIPS.

[6]  Ajith Anil Meera,et al.  Dynamic Expectation Maximization Algorithm for Estimation of Linear Systems with Colored Noise , 2021, Entropy.

[7]  Pau Vilimelis Aceituno,et al.  Credit Assignment in Neural Networks through Deep Feedback Control , 2021, NeurIPS.

[8]  Alexander Kolesnikov,et al.  MLP-Mixer: An all-MLP Architecture for Vision , 2021, NeurIPS.

[9]  Thomas Lukasiewicz,et al.  Reverse Differentiation via Predictive Coding , 2021, AAAI.

[10]  Máté Lengyel,et al.  Contextual inference underlies the learning of sensorimotor repertoires , 2020, Nature.

[11]  Beren Millidge,et al.  Relaxing the Constraints on Predictive Coding Models , 2020, ArXiv.

[12]  Mohamad Sawan,et al.  Analog Circuits to Accelerate the Relaxation Process in the Equilibrium Propagation Algorithm , 2020, 2020 IEEE International Symposium on Circuits and Systems (ISCAS).

[13]  Benjamin F. Grewe,et al.  A Theoretical Framework for Target Propagation , 2020, NeurIPS.

[14]  Beren Millidge,et al.  Predictive Coding Approximates Backprop Along Arbitrary Computation Graphs , 2020, Neural Computation.

[15]  Adam Santoro,et al.  Backpropagation and the brain , 2020, Nature Reviews Neuroscience.

[16]  Surya Ganguli,et al.  A deep learning framework for neuroscience , 2019, Nature Neuroscience.

[17]  Peter C. Humphreys,et al.  Deep Learning without Weight Transport , 2019, NeurIPS.

[18]  James C. R. Whittington,et al.  Theories of Error Back-Propagation in the Brain , 2019, Trends in Cognitive Sciences.

[19]  Yoshua Bengio,et al.  Dendritic cortical microcircuits approximate the backpropagation algorithm , 2018, NeurIPS.

[20]  Georg B. Keller,et al.  Predictive Processing: A Canonical Cortical Computation , 2018, Neuron.

[21]  Bryan Catanzaro,et al.  Large Scale Language Modeling: Converging on 40GB of Text in Four Hours , 2018, 2018 30th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD).

[22]  N. Harper,et al.  Sensory cortex is optimized for prediction of future input , 2017, bioRxiv.

[23]  Roland Vollgraf,et al.  Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms , 2017, ArXiv.

[24]  D. Hassabis,et al.  Neuroscience-Inspired Artificial Intelligence , 2017, Neuron.

[25]  Alexander Attinger,et al.  Visuomotor Coupling Shapes the Functional Development of Mouse Visual Cortex , 2017, Cell.

[26]  Rafal Bogacz,et al.  An Approximation of the Error Backpropagation Algorithm in a Predictive Coding Network with Local Hebbian Synaptic Plasticity , 2017, Neural Computation.

[27]  Surya Ganguli,et al.  Continual Learning Through Synaptic Intelligence , 2017, ICML.

[28]  Wieland Brendel,et al.  Learning to represent signals spike by spike , 2017, PLoS Comput. Biol..

[29]  Wulfram Gerstner,et al.  Predicting non-linear dynamics by stable local learning in a recurrent spiking neural network , 2017, eLife.

[30]  Víctor Soto,et al.  An urn model for majority voting in classification ensembles , 2016, NIPS.

[31]  Colin J. Akerman,et al.  Random synaptic feedback weights support error backpropagation for deep learning , 2016, Nature Communications.

[32]  Timothy P Lillicrap,et al.  Towards deep learning with segregated dendrites , 2016, eLife.

[33]  J. Schulman,et al.  OpenAI Gym , 2016, ArXiv.

[34]  John J. Hopfield,et al.  Dense Associative Memory for Pattern Recognition , 2016, NIPS.

[35]  Yoshua Bengio,et al.  Equilibrium Propagation: Bridging the Gap between Energy-Based Models and Backpropagation , 2016, Front. Comput. Neurosci..

[36]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[37]  Keiron O'Shea,et al.  An Introduction to Convolutional Neural Networks , 2015, ArXiv.

[38]  Yoshua Bengio,et al.  Early Inference in Energy-Based Models Approximates Back-Propagation , 2015, ArXiv.

[39]  Yoshua Bengio,et al.  STDP as presynaptic activity times rate of change of postsynaptic activity , 2015, 1509.05936.

[40]  Alborz Geramifard,et al.  RLPy: a value-function-based reinforcement learning framework for education and research , 2015, J. Mach. Learn. Res..

[41]  Georg B. Keller,et al.  Learning Enhances Sensory and Multiple Non-sensory Representations in Primary Visual Cortex , 2015, Neuron.

[42]  Marc G. Bellemare,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[43]  Christian Szegedy,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[44]  Yoshua Bengio,et al.  Difference Target Propagation , 2014, ECML/PKDD.

[45]  Yoshua Bengio,et al.  How Auto-Encoders Could Provide Credit Assignment in Deep Networks via Target Propagation , 2014, ArXiv.

[46]  Ha Hong,et al.  Performance-optimized hierarchical models predict neural responses in higher visual cortex , 2014, Proceedings of the National Academy of Sciences.

[47]  A. Bifet,et al.  A survey on concept drift adaptation , 2014, ACM Comput. Surv..

[48]  Surya Ganguli,et al.  Exact solutions to the nonlinear dynamics of learning in deep linear neural networks , 2013, ICLR.

[49]  Christian K. Machens,et al.  Predictive Coding of Dynamical Variables in Balanced Spiking Networks , 2013, PLoS Comput. Biol..

[50]  Jacek M. Zurada,et al.  Efficiency and Scalability Methods for Computational Intellect , 2013 .

[51]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[52]  Karl J. Friston,et al.  Canonical Microcircuits for Predictive Coding , 2012, Neuron.

[53]  Indr.e vZliobait.e,et al.  Learning under Concept Drift: an Overview , 2010, ArXiv.

[54]  E. J. Collins,et al.  Posterior Weighted Reinforcement Learning with State Uncertainty , 2010, Neural Computation.

[55]  S. Kennerley,et al.  Heterogeneous reward signals in prefrontal cortex , 2010, Current Opinion in Neurobiology.

[56]  Xavier Glorot,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[57]  Karl J. Friston.  The free-energy principle: a unified brain theory? , 2010, Nature Reviews Neuroscience.

[58]  Karl J. Friston.  Hierarchical Models in the Brain , 2008, PLoS Comput. Biol..

[59]  Karl J. Friston,et al.  DEM: A variational treatment of dynamic systems , 2008, NeuroImage.

[60]  E. Procyk,et al.  Behavioral Shifts and Action Valuation in the Anterior Cingulate Cortex , 2008, Neuron.

[61]  Justin Dauwels,et al.  On Variational Message Passing on Factor Graphs , 2007, 2007 IEEE International Symposium on Information Theory.

[62]  J. O'Doherty,et al.  The Role of the Ventromedial Prefrontal Cortex in Abstract State-Based Inference during Decision Making in Humans , 2006, The Journal of Neuroscience.

[63]  Karl J. Friston,et al.  A theory of cortical responses , 2005, Philosophical Transactions of the Royal Society B: Biological Sciences.

[64]  Leslie G. Ungerleider,et al.  A general mechanism for perceptual decision-making in the human brain , 2004, Nature.

[65]  R. O’Reilly,et al.  Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain , 2000 .

[66]  Randall C. O'Reilly,et al.  Biologically Plausible Error-Driven Learning Using Local Activation Differences: The Generalized Recirculation Algorithm , 1996, Neural Computation.

[67]  R. Sutton,et al.  Advances in Neural Information Processing Systems pp MIT Press Generalization in Reinforcement Learning Successful Examples Using Sparse Coarse Coding , 2010 .

[68]  James L. McClelland,et al.  Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. , 1995, Psychological review.

[69]  L. B. Almeida.  A learning rule for asynchronous perceptrons with feedback in a combinatorial environment , 1990 .

[70]  Joachim Diederich,et al.  Artificial neural networks: concept learning , 1990 .

[71]  Fernando J. Pineda,et al.  Dynamics and architecture for neural computation , 1988, J. Complex..

[72]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[73]  Ralph R. Miller,et al.  Recovery of an overshadowed association achieved by extinction of the overshadowing stimulus. , 1985 .

[74]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[75]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[76]  R. Bolles,et al.  A nonassociative aspect of overshadowing , 1981 .

[77]  B. Welford.  Note on a Method for Calculating Corrected Sums of Squares and Products , 1962 .

[78]  Oscar Fontenla-Romero,et al.  Online Machine Learning , 2024, Machine Learning: Foundations, Methodologies, and Applications.

[79]  J. Oswald,et al.  The least-control principle for learning at equilibrium , 2022, ArXiv.

[80]  Thomas Lukasiewicz,et al.  Can the Brain Do Backpropagation? - Exact Implementation of Backpropagation in Predictive Coding Networks , 2020, NeurIPS.

[81]  Joshua B. Tenenbaum,et al.  Human Learning in Atari , 2017, AAAI Spring Symposia.

[82]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[83]  Y. Takane,et al.  Generalized Inverse Matrices , 2011 .

[84]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[85]  R. O’Reilly,et al.  Computational Explorations in Cognitive Neuroscience , 2009 .

[86]  Alexey Tsymbal,et al.  The problem of concept drift: definitions and related work , 2004 .

[87]  Rajesh P. N. Rao,et al.  Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. , 1999 .

[88]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[89]  Andrew W. Moore,et al.  Efficient memory-based learning for robot control , 1990 .

[90]  Michael McCloskey,et al.  Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem , 1989 .

[91]  Fernando J. Pineda,et al.  Generalization of Back propagation to Recurrent and Higher Order Neural Networks , 1987, NIPS.

[92]  Donald E. Knuth,et al.  The Art of Computer Programming, Vol. 2 , 1981 .

[93]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .