论文信息 - Probabilistic Transformers - 字舞流文

Probabilistic Transformers

We show that Transformers are Maximum Posterior Probability estimators for Mixtures of Gaussian Models. This brings a probabilistic point of view to Transformers and suggests extensions to other probabilistic cases.

Javier R. Movellan

[1] Yoshua Bengio,et al. Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[2] D. Rubin,et al. Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[3] Lukasz Kaiser,et al. Attention is All you Need , 2017, NIPS.

[4] Yukun Zhu,et al. Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation , 2020, ECCV.

[5] Andriy Mnih,et al. The Lipschitz Constant of Self-Attention , 2020, ArXiv.

[6] Radu Soricut,et al. Attention that does not Explain Away , 2020, ArXiv.

[7] Zheng Zhang,et al. Disentangled Non-Local Neural Networks , 2020, ECCV.

[8] Geir Kjetil Sandve,et al. Hopfield Networks is All You Need , 2020, ArXiv.