Wavelet kernel function based multiscale LSSVM for elliptic boundary value problems

Abstract Recently, Mehrkanoon and Suykens designed a least square support vector machine (LSSVM) for learning solutions to partial differential equations in [1], where the Gaussian radial basis function is used as the kernel of the LSSVM. The purpose of the present paper is twofold: firstly, we extend the Gaussian kernel to wavelet kernel; and secondly, we propose a multiscale scheme, by noticing the multiscale nature of the wavelet kernel functions. The multiscale algorithm consists of a sequence of residual corrections to the solution of the partial differential equation, in which different scale parameters are employed to accommodate information at different scales. However, the direct computation of the multiscale problem would be difficult due to the mixing of different scales. To resolve this problem, we further introduce a multilevel algorithm, which decompose the multiscale algorithm into multiple levels: on the first level, a coarse data set and a large scale parameter are chosen and the target function is interpolated in this data set to capture the large-scale variations of the target function; next, on the second level, a smaller scale parameter is used to interpolate the residuals on a finer data set, capturing the finer details. The sum of both interpolants obviously better approximates the target function at the data sites on the finer data set. This process can be further applied to finer and finer scales till the anticipated accuracy is achieved. Note that sometimes the correction on the finer scale is necessary only at some local sites, thus an adaptive algorithm with only local corrections on finer scale is introduced. The numerical tests on some linear second order elliptic boundary value problems show the efficiency of the multilevel algorithm and the adaptive algorithm.

[1]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[2]  Ju-Hong Lee,et al.  Comparison of generalization ability on solving differential equations using backpropagation and reformulated radial basis function networks , 2009, Neurocomputing.

[3]  Ingrid Daubechies,et al.  Ten Lectures on Wavelets , 1992 .

[4]  Harold H. Szu,et al.  Neural network adaptive wavelets for signal representation and classification , 1992 .

[5]  Mohsen Hayati,et al.  Multilayer perceptron neural networks with novel unsupervised training method for numerical solution of the partial differential equations , 2009, Appl. Soft Comput..

[6]  Holger Wendland,et al.  Kernel techniques: From machine learning to meshless methods , 2006, Acta Numerica.

[7]  M. H. Kantli,et al.  A new wavelet multigrid method for the numerical solution of elliptic type differential equations , 2017 .

[8]  Li Zhang,et al.  Wavelet support vector machine , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[9]  Johan A. K. Suykens,et al.  Learning solutions to partial differential equations using LS-SVM , 2015, Neurocomputing.

[10]  Anouar Ben Mabrouk,et al.  A Wavelet Support Vector Machine Coupled Method for Time Series Prediction , 2008, Int. J. Wavelets Multiresolution Inf. Process..

[11]  J. Mercer.  Functions of Positive and Negative Type, and their Connection with the Theory of Integral Equations , 1909 .

[12]  Roman Neruda,et al.  Learning methods for radial basis function networks , 2005, Future Gener. Comput. Syst..

[13]  Ethem Alpaydin,et al.  Multiple Kernel Learning Algorithms , 2011, J. Mach. Learn. Res..

[14]  Wang Xianfang,et al.  Multi-scale Least Square Wavelet Support Vector Machine , 2012, 2012 24th Chinese Control and Decision Conference (CCDC).

[15]  Mark A Fleming,et al.  Meshless methods: An overview and recent developments , 1996 .

[16]  Bernhard Schölkopf,et al.  The connection between regularization operators and support vector kernels , 1998, Neural Networks.

[17]  Du Zhiyong,et al.  An improved LSSVM based on multi-scale wavelet kernel function , 2012, Proceedings of the 31st Chinese Control Conference.

[18]  Dimitrios I. Fotiadis,et al.  Artificial neural networks for solving ordinary and partial differential equations , 1997, IEEE Trans. Neural Networks.

[19]  Chuanhou Gao,et al.  Design of a multiple kernel learning algorithm for LS-SVM by convex programming , 2011, Neural Networks.

[20]  Holger Wendland,et al.  RBF Multiscale Collocation for Second Order Elliptic Boundary Value Problems , 2013, SIAM J. Numer. Anal..

[21]  Andrew J. Meade,et al.  Solution of nonlinear ordinary differential equations by feedforward neural networks , 1994 .

[22]  Qinghua Zhang,et al.  Wavelet networks , 1992, IEEE Trans. Neural Networks.

[23]  Nicholas K.-R. Kevlahan,et al.  An adaptive multilevel wavelet collocation method for elliptic problems , 2005 .

[24]  Stephen A. Billings,et al.  Generalized multiscale radial basis function networks , 2007, Neural Networks.

[25]  Gunnar Rätsch,et al.  A General and Efficient Multiple Kernel Learning Algorithm , 2005, NIPS.