Solving Bilinear Inverse Problems using Deep Generative Priors

This paper proposes a new framework to handle the bilinear inverse problems (BIPs): recover $\boldsymbol{w}$, and $\boldsymbol{x}$ from the measurements of the form $\boldsymbol{y} = \mathcal{A}(\boldsymbol{w},\boldsymbol{x})$, where $\mathcal{A}$ is a bilinear operator. The recovery problem of the unknowns $\boldsymbol{w}$, and $\boldsymbol{x}$ can be formulated as a non-convex program. A general strategy is proposed to turn the ill-posed BIP to a relatively well-conditioned BIP by imposing a structural assumption that $\boldsymbol{w}$, and $\boldsymbol{x}$ are members of some classes $\mathcal{W}$, and $\mathcal{X}$, respectively, that are parameterized by unknown latent low-dimensional features. We learn functions mapping from the hidden feature space to the ambient space for each class using generative models. The resulting reduced search space of the solution enables a simple alternating gradient descent scheme to yield promising result in solving the non-convex BIP. To demonstrate the performance of our algorithm, we choose an important BIP; namely, blind image deblurring as a motivating application. We show through extensive experiments that this technique shows promising results in deblurring images of real datasets and is also robust to noise perturbations.

[1]  William T. Freeman,et al.  Removing camera shake from a single photograph , 2006, SIGGRAPH 2006.

[2]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[3]  Jian-Feng Cai,et al.  Blind motion deblurring from a single image using sparse approximation , 2009, CVPR.

[4]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[5]  Tae Hyun Kim,et al.  Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Jan Kotera,et al.  Convolutional Neural Networks for Direct Text Deblurring , 2015, BMVC.

[7]  Deqing Sun,et al.  Blind Image Deblurring Using Dark Channel Prior , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Olivier Eches,et al.  A Bilinear–Bilinear Nonnegative Matrix Factorization Method for Hyperspectral Unmixing , 2014, IEEE Geoscience and Remote Sensing Letters.

[9]  Ming-Hsuan Yang,et al.  Single image deblurring with adaptive dictionary learning , 2010, 2010 IEEE International Conference on Image Processing.

[10]  Thomas Strohmer,et al.  Regularized Gradient Descent: A Nonconvex Recipe for Fast Joint Blind Deconvolution and Demixing , 2017, ArXiv.

[11]  Xiaochun Cao,et al.  Image Deblurring via Enhanced Low-Rank Prior , 2016, IEEE Transactions on Image Processing.

[12]  Ali Ahmed,et al.  BranchHull: Convex bilinear inversion from the entrywise product of signals with known signs , 2017, Applied and Computational Harmonic Analysis.

[13]  Frédo Durand,et al.  Understanding and evaluating blind deconvolution algorithms , 2009, CVPR.

[14]  Ian J. Goodfellow,et al.  NIPS 2016 Tutorial: Generative Adversarial Networks , 2016, ArXiv.

[15]  Tony F. Chan,et al.  Total variation blind deconvolution , 1998, IEEE Trans. Image Process..

[16]  Justin K. Romberg,et al.  Blind Deconvolution Using Convex Programming , 2012, IEEE Transactions on Information Theory.

[17]  Bernhard Schölkopf,et al.  Learning to Deblur , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Yonina C. Eldar,et al.  Phase Retrieval: An Overview of Recent Developments , 2015, ArXiv.

[19]  Guili Liu,et al.  Motion blur kernel estimation via salient edges and low rank prior , 2014, 2014 IEEE International Conference on Multimedia and Expo (ICME).

[20]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[21]  Andrew Y. Ng,et al.  Reading Digits in Natural Images with Unsupervised Feature Learning , 2011 .

[22]  Thomas S. Huang,et al.  Sparse representation based blind image deblurring , 2011, 2011 IEEE International Conference on Multimedia and Expo.

[23]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[24]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[25]  Wojciech Zaremba,et al.  Improved Techniques for Training GANs , 2016, NIPS.

[26]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[27]  Deqing Sun,et al.  Learning to Super-Resolve Blurry Face and Text Images , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[28]  Alexandros G. Dimakis,et al.  Compressed Sensing using Generative Models , 2017, ICML.

[29]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[30]  Yann LeCun,et al.  The mnist database of handwritten digits , 2005 .

[31]  XuYi,et al.  Image smoothing via L0 gradient minimization , 2011 .

[32]  Pascal Frossard,et al.  Dictionary Learning , 2011, IEEE Signal Processing Magazine.

[33]  Ali Ahmed Low-rank matrix recovery: blind deconvolution and efficient sampling of correlated signals , 2013 .

[34]  Yonina C. Eldar,et al.  Phase Retrieval via Matrix Completion , 2011, SIAM Rev..

[35]  Thomas Strohmer,et al.  Blind Deconvolution Meets Blind Demixing: Algorithms and Performance Bounds , 2015, IEEE Transactions on Information Theory.