Amortized Variational Compressive Sensing

The goal of statistical compressive sensing is to efficiently acquire and reconstruct high-dimensional signals with much fewer measurements, given access to a finite set of training signals from the underlying domain being sensed. We present a novel algorithmic framework based on autoencoders that jointly learns the acquisition (a.k.a. encoding) and recovery (a.k.a. decoding) functions while implicitly modeling domain structure. Our learning objective maximizes a variational lower bound to the mutual information between the signal and the measurements. Empirically, we show 20− 46% improvement in reconstruction accuracies over competing approaches on the MNIST dataset for the same number of measurements.

[1]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[2]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[3]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[4]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[5]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[6]  Hyun Sung Chang,et al.  Learning Compressed Sensing , 2007 .

[7]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[8]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[9]  Thomas Strohmer,et al.  High-Resolution Radar via Compressed Sensing , 2008, IEEE Transactions on Signal Processing.

[10]  P. Bickel,et al.  SIMULTANEOUS ANALYSIS OF LASSO AND DANTZIG SELECTOR , 2008, 0801.1095.

[11]  Guillermo Sapiro,et al.  Statistical Compressed Sensing of Gaussian Mixture Models , 2011, IEEE Transactions on Signal Processing.

[12]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[13]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[14]  Alexandros G. Dimakis,et al.  Compressed Sensing using Generative Models , 2017, ICML.