Autoencoder and Its Various Variants

The concept of autoencoder was originally proposed by LeCun in 1987, early works on autoencoder were used for dimensionality reduction or feature learning. Recently, with the popularity of deep learning research, autoencoder has been brought to the forefront of generative modeling. Many variants of autoencoder have been proposed by different researchers and have been successfully applied in many fields, such as computer vision, speech recognition and natural language processing. In this paper, we present a comprehensive survey on autoencoder and its various variants. Furthermore, we also present the lineage of the surveyed autoencoders. This paper can provide researchers engaged in related works with very valuable help.

[1]  Yann LeCun.  PhD thesis: Modeles connexionnistes de l'apprentissage (connectionist learning models) , 1987 .

[2]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[3]  Guang-Bin Huang,et al.  Trends in extreme learning machines: A review , 2015, Neural Networks.

[4]  Eder Santana,et al.  Information Theoretic-Learning auto-encoder , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[5]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[6]  David M. Blei,et al.  Variational Inference: A Review for Statisticians , 2016, ArXiv.

[7]  Antonia Creswell,et al.  Denoising Adversarial Autoencoders , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[9]  Jürgen Schmidhuber,et al.  Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction , 2011, ICANN.

[10]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[11]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[12]  Alan F. Murray,et al.  International Joint Conference on Neural Networks , 1993 .

[13]  Sebastian Nowozin,et al.  Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks , 2017, ICML.

[14]  Matt J. Kusner,et al.  Grammar Variational Autoencoder , 2017, ICML.

[15]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[16]  Geoffrey E. Hinton,et al.  Autoencoders, Minimum Description Length and Helmholtz Free Energy , 1993, NIPS.

[17]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[18]  Tom White,et al.  Generative Adversarial Networks: An Overview , 2017, IEEE Signal Processing Magazine.

[19]  Dianhui Wang,et al.  Extreme learning machines: a survey , 2011, Int. J. Mach. Learn. Cybern..

[20]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[21]  H. Bourlard,et al.  Auto-association by multilayer perceptrons and singular value decomposition , 1988, Biological Cybernetics.

[22]  Pascal Vincent,et al.  Contractive Auto-Encoders: Explicit Invariance During Feature Extraction , 2011, ICML.

[23]  David Duvenaud,et al.  Inference Suboptimality in Variational Autoencoders , 2018, ICML.

[24]  Pascal Vincent,et al.  Higher Order Contractive Auto-Encoder , 2011, ECML/PKDD.

[25]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[26]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.