Onsager-Corrected Deep Networks for Sparse Linear Inverse Problems

Deep learning has gained great popularity due to its widespread success on many inference problems. We consider the application of deep learning to the sparse linear inverse pr oblem encountered in compressive sensing, where one seeks to reco ver a sparse signal from a few noisy linear measurements. In this p aper, we propose two novel neural-network architectures that dec ouple prediction errors across layers in the same way that the appr oximate message passing (AMP) algorithms decouple them across iterations: through Onsager correction. We show numericallythat our “learned AMP” network significantly improves upon Grego r and LeCun’s “learned ISTA” when both use soft-thresholding shrinkage. We then show that additional improvements resul t from jointly learning the shrinkage functions together with the linear transforms. Finally, we propose a network design ins pired by an unfolding of the recently proposed “vector AMP” (VAMP) algorithm, and show that it outperforms all previously considered networks. Interestingly, the linear transforms and shrinkage functions prescribed by VAMP coincide with the values learned through backpropagation, yielding an intuitive explanation for the design of this deep network.