Multiscale Support Vector Approach for Solving Ill-Posed Problems

Based on the use of compactly supported radial basis functions, we extend in this paper the support vector approach to a multiscale support vector approach (MSVA) scheme for approximating the solution of a moderately ill-posed problem on bounded domain. The Vapnik’s $$\epsilon $$ϵ-intensive function is adopted to replace the standard $$l^2$$l2 loss function in using the regularization technique to reduce the error induced by noisy data. Convergence proof for the case of noise-free data is then derived under an appropriate choice of the Vapnik’s cut-off parameter and the regularization parameter. For noisy data case, we demonstrate that a corresponding choice for the Vapnik’s cut-off parameter gives the same order of error estimate as both the a posteriori strategy based on discrepancy principle and the noise-free a priori strategy. Numerical examples are constructed to verify the efficiency of the proposed MSVA approach and the effectiveness of the parameter choices.

[1]  Robert Schaback,et al.  The missing Wendland functions , 2011, Adv. Comput. Math..

[2]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[3]  Reinhold Schneider,et al.  Self–regularization by projection for noisy pseudodifferential equations of negative order , 2003, Numerische Mathematik.

[4]  R. L. Hardy Multiquadric equations of topography and other irregular surfaces , 1971 .

[5]  Holger Wendland,et al.  Multiscale approximation for functions in arbitrary Sobolev spaces by scaled radial basis functions on the unit sphere , 2012 .

[6]  A. K. Louis,et al.  Sobolev error estimates and a priori parameter selection for semi-discrete Tikhonov regularization , 2009 .

[7]  Charles A. Micchelli,et al.  Learning the Kernel Function via Regularization , 2005, J. Mach. Learn. Res..

[8]  Holger Wendland,et al.  Multiscale analysis in Sobolev spaces on bounded domains with zero boundary values , 2013 .

[9]  Saburou Saitoh,et al.  Theory of Reproducing Kernels and Its Applications , 1988 .

[10]  Carsten Franke,et al.  Solving partial differential equations by collocation using radial basis functions , 1998, Appl. Math. Comput..

[11]  Holger Wendland,et al.  Piecewise polynomial, positive definite and compactly supported radial functions of minimal degree , 1995, Adv. Comput. Math..

[12]  G. Wahba Spline models for observational data , 1990 .

[13]  Holger Wendland,et al.  Multiscale Analysis in Sobolev Spaces on the Sphere , 2010, SIAM J. Numer. Anal..

[14]  A Kirsch,et al.  The Backus-Gilbert method , 1988 .

[15]  Andreas Rieder,et al.  A wavelet multilevel method for ill-posed problems stabilized by Tikhonov regularization , 1997 .

[16]  Quoc Thong Le Gia,et al.  Multiscale methods with compactly supported radial basis functions for the Stokes problem on bounded domains , 2012, Advances in Computational Mathematics.

[17]  Benny Y. C. Hon,et al.  Compactly supported radial basis functions for shallow water equations , 2002, Appl. Math. Comput..

[18]  Holger Wendland,et al.  Meshless Collocation: Error Estimates with Application to Dynamical Systems , 2007, SIAM J. Numer. Anal..

[19]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[20]  Shuai Lu,et al.  Multiscale analysis for ill-posed problems with semi-discrete Tikhonov regularization , 2012 .

[21]  J. Cooper SINGULAR INTEGRALS AND DIFFERENTIABILITY PROPERTIES OF FUNCTIONS , 1973 .

[22]  Masahiro Yamamoto,et al.  Tikhonov Regularization by a Reproducing Kernel Hilbert Space for the Cauchy Problem for an Elliptic Equation , 2008, SIAM J. Sci. Comput..

[23]  H. Engl,et al.  Regularization of Inverse Problems , 1996 .

[24]  Otmar Scherzer,et al.  An iterative multi level algorithm for solving nonlinear ill–posed problems , 1998, Numerische Mathematik.

[25]  Yuesheng Xu,et al.  A multilevel augmentation method for solving ill-posed operator equations , 2006 .

[26]  Mario Bertero,et al.  Linear inverse problems with discrete data. I. General formulation and singular system analysis , 1985 .

[27]  N. Aronszajn Theory of Reproducing Kernels. , 1950 .

[28]  Holger Wendland,et al.  Multiscale RBF collocation for solving PDEs on spheres , 2012, Numerische Mathematik.

[29]  Sergei V. Pereverzev,et al.  The balancing principle in solving semi-discrete inverse problems in Sobolev scales by Tikhonov method , 2012 .

[30]  Fred J. Hickernell,et al.  Radial basis function approximations as smoothing splines , 1999, Appl. Math. Comput..

[31]  Tom Lyche,et al.  Mathematical Methods for Curves and Surfaces , 2016, Lecture Notes in Computer Science.

[32]  Ting Wei,et al.  Backus-Gilbert algorithm for the Cauchy problem of the Laplace equation , 2001 .

[33]  Martin Burger,et al.  Training neural networks with noisy data as an ill-posed problem , 2000, Adv. Comput. Math..

[34]  Bernhard Schölkopf,et al.  Learning with kernels , 2001 .

[35]  Quoc Thong Le Gia,et al.  Multiscale methods with compactly supported radial basis functions for Galerkin approximation of elliptic PDEs , 2012, 1211.1431.

[36]  Holger Wendland,et al.  Scattered Data Approximation: Conditionally positive definite functions , 2004 .

[37]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[38]  Robert Schaback,et al.  Solvability of partial differential equations by meshless kernel methods , 2008, Adv. Comput. Math..

[39]  Jun Zou,et al.  A multilevel model correction method for parameter identification , 2007 .

[40]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[41]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[42]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[43]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[44]  E. Stein Singular Integrals and Di?erentiability Properties of Functions , 1971 .

[45]  Christian Rieger,et al.  Deterministic Error Analysis of Support Vector Regression and Related Regularized Kernel Methods , 2009, J. Mach. Learn. Res..

[46]  E. Kansa Application of Hardy's multiquadric interpolation to hydrodynamics , 1985 .

[47]  Holger Wendland,et al.  Multiscale analysis in Sobolev spaces on bounded domains , 2010, Numerische Mathematik.

[48]  Sergei V. Pereverzyev,et al.  Regularized collocation method for Fredholm integral equations of the first kind , 2007, J. Complex..

[49]  Benny Y. C. Hon,et al.  Discretized Tikhonov regularization by reproducing kernel Hilbert space for backward heat conduction problem , 2011, Adv. Comput. Math..

[50]  A. Louis Inverse und schlecht gestellte Probleme , 1989 .

[51]  Jochen Krebs,et al.  Support vector regression for the solution of linear integral equations , 2011 .

[52]  Sergei V. Pereverzev,et al.  Regularization Theory for Ill-Posed Problems: Selected Topics , 2013 .