论文信息 - A remark on a paper of Krotov and Hopfield [arXiv: 2008.06996]

A remark on a paper of Krotov and Hopfield [arXiv: 2008.06996]

In their recent paper titled “Large Associative Memory Problem in Neurobiology and Machine Learning” [1] the authors gave a biologically plausible microscopic theory from which one can recover many dense associative memory models discussed in the literature. We show that the layers of the recent ”MLP-mixer” [2] as well as the essentially equivalent model in [3] are amongst them.

Fei Tang | Michael Kopp | Michael Kopp | Fei Tang

[1] Luke Melas-Kyriazi,et al. Do You Even Need Attention? A Stack of Feed-Forward Layers Does Surprisingly Well on ImageNet , 2021, ArXiv.

[2] Alexander Kolesnikov,et al. MLP-Mixer: An all-MLP Architecture for Vision , 2021, NeurIPS.

[3] Dmitry Krotov,et al. Large Associative Memory Problem in Neurobiology and Machine Learning , 2020, ArXiv.

[4] John J. Hopfield,et al. Dense Associative Memory for Pattern Recognition , 2016, NIPS.

[5] Geir Kjetil Sandve,et al. Hopfield Networks is All You Need , 2020, ArXiv.