Convex and Differentiable Formulation for Inverse Problems in Hilbert Spaces with Nonlinear Clipping Effects

Wepropose a useful formulation for ill-posed inverse problems in Hilbert spaces with nonlinear clipping effects. Ill-posed inverse problems are often formulated as optimization problems, and nonlinear clipping effects may cause nonconvexity or nondifferentiability of the objective functions in the case of commonly used regularized least squares. To overcome these difficulties, we present a tractable formulation in which the objective function is convex and differentiable with respect to optimization variables, on the basis of the Bregman divergence associated with the primitive function of the clipping function. By using this formulation in combination with the representer theorem, we need only to deal with a finite-dimensional, convex, and differentiable optimization problem, which can be solved by well-established algorithms. We also show two practical examples of inverse problems where our theory can be applied, estimation of band-limited signals and time-harmonic acoustic fields, and evaluate the validity of our theory by numerical simulations. key words: inverse problem, Hilbert space, representer theorem, Bregman divergence, convex optimization

[1]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[2]  Charles A. Micchelli,et al.  Kernels for Multi--task Learning , 2004, NIPS.

[3]  Hiroshi Saruwatari,et al.  Sound Field Recording Using Distributed Microphones Based on Harmonic Analysis of Infinite Order , 2018, IEEE Signal Processing Letters.

[4]  N. Aronszajn Theory of Reproducing Kernels. , 1950 .

[5]  Martin Burger,et al.  Modern regularization methods for inverse problems , 2018, Acta Numerica.

[6]  G. Wahba Spline models for observational data , 1990 .

[7]  Michael Rabadi,et al.  Kernel Methods for Machine Learning , 2015 .

[8]  Patrick Kofod Mogensen,et al.  Optim: A mathematical optimization package for Julia , 2018, J. Open Source Softw..

[9]  Mark A. Poletti,et al.  Three-Dimensional Surround Sound Systems Based on Spherical Harmonics , 2005 .

[10]  Kung Yao,et al.  Applications of Reproducing Kernel Hilbert Spaces-Bandlimited Signal Models , 1967, Inf. Control..

[11]  Mark D. Plumbley,et al.  Fast Iterative Shrinkage for Signal Declipping and Dequantization , 2018, 1812.01540.

[12]  Laurent Jacques,et al.  Consistent iterative hard thresholding for signal declipping , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[13]  Y. Nesterov A method for solving the convex programming problem with convergence rate O(1/k^2) , 1983 .

[14]  A. Kirsch An Introduction to the Mathematical Theory of Inverse Problems , 1996, Applied Mathematical Sciences.

[15]  L. Armijo Minimization of functions having Lipschitz continuous first partial derivatives. , 1966 .

[16]  Yuesheng Xu,et al.  Functional reproducing kernel Hilbert spaces for non-point-evaluation functional data , 2017, Applied and Computational Harmonic Analysis.

[17]  C. Kelley Iterative Methods for Linear and Nonlinear Equations , 1987 .

[18]  Simon J. Godsill,et al.  Statistical Model-Based Approaches to Audio Restoration and Analysis , 2001 .

[19]  C. Vogel Computational Methods for Inverse Problems , 1987 .

[20]  Marc Moonen,et al.  Declipping of Audio Signals Using Perceptual Compressed Sensing , 2013, IEEE Transactions on Audio, Speech, and Language Processing.

[21]  Hiroshi Saruwatari,et al.  Kernel Ridge Regression with Constraint of Helmholtz Equation for Sound Field Interpolation , 2018, 2018 16th International Workshop on Acoustic Signal Enhancement (IWAENC).

[22]  W. Hager,et al.  A SURVEY OF NONLINEAR CONJUGATE GRADIENT METHODS , 2005 .

[23]  M. Unser Sampling-50 years after Shannon , 2000, Proceedings of the IEEE.

[24]  Rémi Gribonval,et al.  Sparsity and Cosparsity for Audio Declipping: A Flexible Non-convex Approach , 2015, LVA/ICA.

[25]  M. Zuhair Nashed,et al.  General sampling theorems for functions in reproducing kernel Hilbert spaces , 1991, Math. Control. Signals Syst..

[26]  Akira Tanaka,et al.  Kernel-Induced Sampling Theorem , 2010, IEEE Transactions on Signal Processing.

[27]  Michael Unser,et al.  A Unifying Representer Theorem for Inverse Problems and Machine Learning , 2019, Foundations of Computational Mathematics.

[28]  Bernhard Schölkopf,et al.  A Generalized Representer Theorem , 2001, COLT/EuroCOLT.

[29]  Bernhard Schölkopf,et al.  The representer theorem for Hilbert spaces: a necessary and sufficient condition , 2012, NIPS.

[30]  Mark D. Plumbley,et al.  Sparse Recovery and Dictionary Learning From Nonlinear Compressive Measurements , 2019, IEEE Transactions on Signal Processing.