Deep Generative Vision as Approximate Bayesian Computation

Probabilistic formulations of inverse graphics have recently been proposed for a variety of 2D and 3D vision problems [15, 12, 14, 9]. These approaches represent visual elements in form of graphics simulators that produce approximate renderings of the visual scenes. Existing approaches either model pixel data or hand-crafted intermediate representations such as edge maps, super-pixels, silhouettes etc. However, the choice of features can drastically affect inference quality and run-time. Recently, deep learning techniques such as Convolutional Neural Networks (CNNs) have demonstrated impressive performance on various tasks such as object recognition and scene pixel labeling, suggesting the superiority of CNN-based features. Encouraged by this findings, we test the ability of CNNs in combination with Approximate Bayesian Computation (ABC) to invert high dimensional generative inverse graphics models from single images. We successfully applied a variant of the probabilistic approximate MCMC algorithm [21] which uses CNN to quantify summary statistics on two real world problems: inferring 3D pose of humans and generative face analysis from single images. Computer Graphics seems to be advancing at a great pace in terms of designing solutions for hard image synthesis problems. Our experiments indicate that the combination of rich probabilistic inverse graphics models and deep learning approaches could utilize such simulators directly to solve the hard inversion problem.