Relations Between Higher Order TV Regularization and Support Vector Regression

We study the connection between higher order total variation (TV) regularization and support vector regression (SVR) with spline kernels in a one-dimensional discrete setting. We prove that the contact problem arising in the tube formulation of the TV minimization problem is equivalent to the SVR problem. Since the SVR problem can be solved by standard quadratic programming methods this provides us with an algorithm for the solution of the contact problem even for higher order derivatives. Our numerical experiments illustrate the approach for various orders of derivatives and show its close relation to corresponding nonlinear diffusion and diffusion–reaction equations.

[1]  C. Vogel,et al.  Analysis of bounded variation penalty methods for ill-posed problems , 1994 .

[2]  Tony F. Chan,et al.  High-Order Total Variation-Based Image Restoration , 2000, SIAM J. Sci. Comput..

[3]  J. Weickert,et al.  Higher Order Variational Methods for Noise Removal in Signals and Images , 2004 .

[4]  G. Wahba.  Spline models for observational data , 1990 .

[5]  C. Vogel.  Computational Methods for Inverse Problems , 1987 .

[6]  Gabriele Steidl,et al.  A Note on the Dual Treatment of Higher-Order Regularization Functionals , 2005, Computing.

[7]  L. Schumaker,et al.  Best Summation Formulae and Discrete Splines , 1973 .

[8]  Thomas Brox,et al.  On the Equivalence of Soft Wavelet Shrinkage, Total Variation Diffusion, Total Variation Regularization, and SIDEs , 2004, SIAM J. Numer. Anal..

[9]  Federico Girosi,et al.  An Equivalence Between Sparse Approximation and Support Vector Machines , 1998, Neural Computation.

[10]  P. Davies,et al.  Local Extremes, Runs, Strings and Multiresolution , 2001 .

[11]  S. Geer,et al.  Locally adaptive regression splines , 1997 .

[12]  Rachid Deriche,et al.  Regularization, Scale-Space, and Edge Detection Filters , 1996, ECCV.

[13]  Arvid Lundervold,et al.  Noise removal using fourth-order partial differential equation with applications to medical magnetic resonance images in space and time , 2003, IEEE Trans. Image Process..

[14]  Otmar Scherzer,et al.  Tube Methods for BV Regularization , 2003, Journal of Mathematical Imaging and Vision.

[15]  Joachim Weickert,et al.  Relations Between Regularization and Diffusion Filtering , 2000, Journal of Mathematical Imaging and Vision.

[16]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[17]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[18]  A. Hero,et al.  A Fast Spectral Method for Active 3D Shape Reconstruction , 2004 .

[19]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[20]  Mostafa Kaveh,et al.  Fourth-order partial differential equations for noise removal , 2000, IEEE Trans. Image Process..

[21]  Alan S. Willsky,et al.  Nonlinear evolution equations as fast and exact solvers of estimation problems , 2005, IEEE Transactions on Signal Processing.

[22]  Marco Loog,et al.  Support Blob Machines. The Sparsification of Linear Scale Space , 2004, ECCV.

[23]  Otmar Scherzer,et al.  Denoising with higher order derivatives of bounded variation and an application to parameter estimation , 2007, Computing.

[24]  A. Chambolle.  Practical, Unified, Motion and Missing Data Treatment in Degraded Video , 2004, Journal of Mathematical Imaging and Vision.

[25]  Otmar Scherzer.  Taut-String Algorithm and Regularization Programs with G-Norm Data Fit , 2005, Journal of Mathematical Imaging and Vision.

[26]  S. Osher,et al.  On the use of the Dual Norms in Bounded Variation Type Regularization , 2006 .

[27]  Yves Meyer,et al.  Oscillating Patterns in Image Processing and Nonlinear Evolution Equations: The Fifteenth Dean Jacqueline B. Lewis Memorial Lectures , 2001 .

[28]  Jiří Matas,et al.  Computer Vision - ECCV 2004 , 2004, Lecture Notes in Computer Science.

[29]  M. Nikolova.  An Algorithm for Total Variation Minimization and Applications , 2004 .