Inverting VAEs for Improved Generative Accuracy

Recent advances in semi-supervised learning with deep generative models have shown promise in generalizing from small labeled datasets (xl,yl) to large unlabeled ones (xu). When the codomain (y) has known structure, a large unfeatured dataset (yu) is potentially available. We develop a parameter-efficient, deep semisupervised generative model for the purpose of exploiting this untapped data source. Empirical results show improved performance in disentangling variable semantics as well as improved discriminative prediction on a new MNIST task.

[1]  Yann LeCun,et al.  The mnist database of handwritten digits , 2005 .

[2]  Phil Blunsom,et al.  Neural Variational Inference for Text Processing , 2015, ICML.

[3]  Daan Wierstra,et al.  Stochastic Backpropagation and Approximate Inference in Deep Generative Models , 2014, ICML.

[4]  George Papandreou,et al.  Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation , 2015, ArXiv.

[5]  Yee Whye Teh,et al.  The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables , 2016, ICLR.

[6]  Ben Poole,et al.  Categorical Reparameterization with Gumbel-Softmax , 2016, ICLR.

[7]  Max Welling,et al.  Semi-supervised Learning with Deep Generative Models , 2014, NIPS.

[8]  Shakir Mohamed,et al.  Variational Inference with Normalizing Flows , 2015, ICML.

[9]  Christopher Burgess,et al.  beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework , 2016, ICLR 2016.

[10]  Mohammad Ghavamzadeh,et al.  Bottleneck Conditional Density Estimation , 2016, ICML.

[11]  David Vázquez,et al.  PixelVAE: A Latent Variable Model for Natural Images , 2016, ICLR.

[12]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[13]  Ole Winther,et al.  Auxiliary Deep Generative Models , 2016, ICML.

[14]  Zhe Gan,et al.  Triangle Generative Adversarial Networks , 2017, NIPS.

[15]  Oriol Vinyals,et al.  Towards Principled Unsupervised Learning , 2015, ArXiv.

[16]  Yoshua Bengio,et al.  A Recurrent Latent Variable Model for Sequential Data , 2015, NIPS.

[17]  Pieter Abbeel,et al.  InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets , 2016, NIPS.

[18]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[19]  Antonio Torralba,et al.  Semi-Supervised Learning in Gigantic Image Collections , 2009, NIPS.