The Latent-Dependent Deep Rendering Model

Generative models that can capture latent variations in data are difficult to design in complex domains such as natural images where there are large numbers of nuisance variables. Given the success of the Convolutional Neural Networks (CNNs) on inference tasks in such domains, we aim to design generative models whose inference corresponds to the CNNs. One such class is the Deep Rendering Model (DRM). The DRM generates images via multiple levels of abstraction, from coarse to fine scale, and introduces a small set of latent variables at each level. However, a number of simplifying assumptions are made in the DRM to derive the CNNs as the bottomup inference algorithm which do not correspond to realistic variation in natural images. For instance, the latent variables at different scales of the DRM are assumed to be independent. We propose an extension to the DRM, termed the LatentDependent Deep Rendering Model (LD-DRM), that enforces dependencies among the latent variables via a parametrized joint prior distribution. This joint prior yields a new form of regularization for training the CNNs—the Rendering Path Normalization (RPN). Under the LD-DRM, we obtain consistent estimators for unsupervised/semisupervised learning tasks and derive generalization bounds. Our bound suggests that the RPN regularization helps improve generalization, an observation that is corroborated in practice. In our experiments, the LD-DRM either beats or matches the state of art on popular benchmarks including SVHN, CIFAR10, and CIFAR100 on both semi-supervised and supervised learning tasks. Equal contribution as first authors , Equal contribution as last authors University of California at Berkeley, Berkeley, USA Rice University, Houston, USA Amazon AI, Palo Alto, USA Baylor College of Medicine, Houston, USA California Institute of Technology, Pasadena, USA. Correspondence to: Nhat Ho <minhnhat@berkeley.edu>, Tan Nguyen <mn15@rice.edu>. Presented at the ICML 2018 workshop on Theoretical Foundations and Applications of Deep Generative Models