Learned Image Deblurring by Unfolding a Proximal Interior Point Algorithm

Image restoration is frequently addressed by resorting to variational methods which account for some prior knowledge about the solution. The success of these methods, however, heavily depends on the estimation of a set of hyperparameters. Deep learning architectures are, on the contrary, very generic and efficient, but they offer limited control over their output. In this paper, we present iRestNet, a neural network architecture which combines the benefits of both approaches. iRestNet is obtained by unfolding a proximal interior point algorithm. This enables enforcing hard constraints on the pixel range of the restored image thanks to a logarithmic barrier strategy, without requiring any parameter setting. Explicit expressions for the involved proximity operator, and its differential, are derived, which allows training iRestNet with gradient descent and backpropagation. Numerical experiments on image deblurring show that the proposed approach provides good image quality results compared to state-of-the-art variational and machine learning methods.

[1]  Yann LeCun,et al.  Learning Fast Approximations of Sparse Coding , 2010, ICML.

[2]  Bernhard Schölkopf,et al.  A Machine Learning Approach for Non-blind Image Deconvolution , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  A. Iusem.  On the convergence properties of the projected gradient method for convex optimization , 2003 .

[4]  Jian Sun,et al.  Deep ADMM-Net for Compressive Sensing MRI , 2016, NIPS.

[5]  Mohamed-Jalal Fadili,et al.  Stein Unbiased GrAdient estimator of the Risk (SUGAR) for Multiple Parameter Selection , 2014, SIAM J. Imaging Sci..

[6]  Bernhard Schölkopf,et al.  Learning to Deblur , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Michael Unser,et al.  Deep Convolutional Neural Network for Inverse Problems in Imaging , 2016, IEEE Transactions on Image Processing.

[8]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[9]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[10]  Michael Unser,et al.  Convolutional Neural Networks for Inverse Problems in Imaging: A Review , 2017, IEEE Signal Processing Magazine.

[11]  Nicolas Couellan,et al.  Interior Point Methods for Supervised Training of Artificial Neural Networks with Bounded Weights , 1997 .

[12]  Émilie Chouzenoux,et al.  PIPA: A New Proximal Interior Point Algorithm for Large-Scale Convex Optimization , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[13]  Jonathan Le Roux,et al.  Deep Unfolding: Model-Based Inspiration of Novel Deep Architectures , 2014, ArXiv.

[14]  Ce Liu,et al.  Deep Convolutional Neural Network for Image Deconvolution , 2014, NIPS.

[15]  Valérie R. Wajs,et al.  A variational formulation for frame-based inverse problems , 2007 .

[16]  Jean-François Aujol,et al.  Some First-Order Algorithms for Total Variation Based Image Restoration , 2009, Journal of Mathematical Imaging and Vision.

[17]  Alexander Kaplan,et al.  Proximal Methods in View of Interior-Point Strategies , 1995, Universität Trier, Mathematik/Informatik, Forschungsbericht.

[18]  Panagiotis T. Krasopoulos,et al.  An Interior Point Recurrent Neural Network for Convex Optimization Problems , 2014 .

[19]  Yoshua Bengio,et al.  Série Scientifique Scientific Series Incorporating Second-order Functional Knowledge for Better Option Pricing Incorporating Second-order Functional Knowledge for Better Option Pricing , 2022 .

[20]  Frédo Durand,et al.  Understanding and evaluating blind deconvolution algorithms , 2009, CVPR.

[21]  Émilie Chouzenoux,et al.  Geometry-Texture Decomposition/Reconstruction Using a Proximal Interior Point Algorithm , 2018, 2018 IEEE 10th Sensor Array and Multichannel Signal Processing Workshop (SAM).

[22]  Yair Weiss,et al.  From learning models of natural image patches to whole image restoration , 2011, 2011 International Conference on Computer Vision.

[23]  Wangmeng Zuo,et al.  Learning Deep CNN Denoiser Prior for Image Restoration , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Atilla Elçi,et al.  Image denoising by median filter in wavelet domain , 2017, ArXiv.

[25]  Patrick L. Combettes,et al.  Deep Neural Network Structures Solving Variational Inequalities , 2018, Set-Valued and Variational Analysis.

[26]  Bernard Ghanem,et al.  ISTA-Net: Interpretable Optimization-Inspired Deep Network for Image Compressive Sensing , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[27]  Xiaohan Chen,et al.  Theoretical Linear Convergence of Unfolded ISTA and its Practical Weights and Thresholds , 2018, NeurIPS.

[28]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[29]  Otmar Scherzer,et al.  The use of Morozov's discrepancy principle for Tikhonov regularization for solving nonlinear ill-posed problems , 1993, Computing.

[30]  J. Zico Kolter,et al.  OptNet: Differentiable Optimization as a Layer in Neural Networks , 2017, ICML.