Signal recovery from Pooling Representations

In this work we compute lower Lipschitz bounds of lp pooling operators for p = 1, 2, ∞ as well as lp pooling operators preceded by half-rectification layers. These give sufficient conditions for the design of invertible neural network layers. Numerical experiments on MNIST and image patches confirm that pooling layers can be inverted with phase recovery algorithms. Moreover, the regularity of the inverse pooling, controlled by the lower Lipschitz constant, is empirically verified with a nearest neighbor regression.

[1]  R. Gerchberg.  A practical algorithm for the determination of phase from image and diffraction plane pictures , 1972 .

[2]  Aapo Hyvärinen,et al.  A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images , 2001, Vision Research.

[3]  Brendan J. Frey,et al.  Probabilistic Inference of Speech Signals from Phaseless Spectrograms , 2003, NIPS.

[4]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[5]  R. Balan,et al.  On signal reconstruction without phase , 2006 .

[6]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[7]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[8]  R. Fergus,et al.  Learning invariant features through topographic filter maps , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[10]  Stéphane Mallat,et al.  Group Invariant Scattering , 2011, ArXiv.

[11]  Emmanuel J. Candès,et al.  PhaseLift: Exact and Stable Signal Recovery from Magnitude Measurements via Convex Programming , 2011, ArXiv.

[12]  Allen Y. Yang,et al.  CPRL -- An Extension of Compressive Sensing to the Phase Retrieval Problem , 2012, NIPS.

[13]  Stéphane Mallat,et al.  Invariant Scattering Convolution Networks , 2012, IEEE transactions on pattern analysis and machine intelligence.

[14]  P. Casazza,et al.  Phase Retrieval By Projections , 2013, 1305.6226.

[15]  Yoshua Bengio,et al.  Maxout Networks , 2013, ICML.

[16]  Yang Wang,et al.  Invertibility and Robustness of Phaseless Reconstruction , 2013, Applied and Computational Harmonic Analysis.

[17]  Yonina C. Eldar,et al.  On conditions for uniqueness in sparse phase retrieval , 2013, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[18]  Alexandre d'Aspremont,et al.  Phase recovery, MaxCut and complex semidefinite programming , 2012, Math. Program..