The emergence of a concept in shallow neural networks

Elena Agliari, Francesco Alemanno, Adriano Barra, and Giordano De Marzo Dipartimento di Matematica, Sapienza Università di Roma, P.le A. Moro 5, 00185, Rome, Italy. Dipartimento di Matematica e Fisica, Università del Salento, Campus Ecotekne, via Monteroni, Lecce 73100, Italy. Istituto Nazionale di Fisica Nucleare, Sezione di Lecce, Campus Ecotekne, via Monteroni, Lecce 73100, Italy. Dipartimento di Fisica, Sapienza Università di Roma, P.le A. Moro 5, 00185, Rome, Italy. and Centro Ricerche Enrico Fermi, Via Panisperna 89a, 00184 Rome, Italy.

[1]  Elena Agliari,et al.  Multitasking associative networks. , 2011, Physical review letters.

[2]  C. Gross.  Genealogy of the “Grandmother Cell” , 2002, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[3]  Cristopher Moore,et al.  The Nature of Computation , 2011 .

[4]  Andrea Montanari,et al.  A mean field view of the landscape of two-layer neural networks , 2018, Proceedings of the National Academy of Sciences.

[5]  M. Mézard,et al.  Analytic and Algorithmic Solution of Random Satisfiability Problems , 2002, Science.

[6]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[7]  Adriano Barra,et al.  On the equivalence of Hopfield Networks and Restricted Boltzmann Machines , 2011, ArXiv.

[8]  Sompolinsky,et al.  Storing infinite numbers of patterns in a spin-glass model of neural networks. , 1985, Physical review letters.

[9]  Peter Sollich,et al.  Theory of Neural Information Processing Systems , 2005 .

[10]  Giancarlo Fissore,et al.  Thermodynamics of Restricted Boltzmann Machines and Related Learning Dynamics , 2018, Journal of Statistical Physics.

[11]  Elena Agliari,et al.  Machine learning and statistical physics: preface , 2020, Journal of Physics A: Mathematical and Theoretical.

[12]  José F. Fontanari,et al.  Generalization in a Hopfield network , 1990 .

[13]  Haiping Huang.  Variational mean-field theory for training restricted Boltzmann machines with binary synapses. , 2020, Physical review. E.

[14]  Rémi Monasson,et al.  Emergence of Compositional Representations in Restricted Boltzmann Machines , 2016, Physical review letters.

[15]  Florent Krzakala,et al.  Statistical physics-based reconstruction in compressed sensing , 2011, ArXiv.

[16]  Elena Agliari,et al.  On the effective initialisation for restricted Boltzmann machines via duality with Hopfield model , 2021, Neural Networks.

[17]  Geoffrey E. Hinton.  Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[18]  M. Mézard.  Mean-field message-passing equations in the Hopfield model and its generalizations. , 2016, Physical review. E.

[19]  F. Guerra.  Broken Replica Symmetry Bounds in the Mean Field Spin Glass Model , 2002, cond-mat/0205123.

[20]  Elena Agliari,et al.  Neural Networks with a Redundant Representation: Detecting the Undetectable. , 2019, Physical review letters.

[21]  Geoffrey E. Hinton,et al.  Deep Boltzmann Machines , 2009, AISTATS.

[22]  Jeffrey S. Bowers,et al.  What is a grandmother cell? And how would you know if you found one? , 2011, Connect. Sci..

[23]  Elena Agliari,et al.  Boltzmann Machines as Generalized Hopfield Networks: A Review of Recent Results and Outlooks , 2020, Entropy.

[24]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[25]  Haiping Huang,et al.  Statistical physics of unsupervised learning with prior knowledge in neural networks , 2019, Physical review letters.

[26]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[27]  Sompolinsky,et al.  Statistical mechanics of learning from examples. , 1992, Physical review. A, Atomic, molecular, and optical physics.

[28]  Haiping Huang,et al.  Statistical mechanics of unsupervised feature learning in a restricted Boltzmann machine with binary synapses , 2016, ArXiv.