Sparse radial basis function approximation with spatially variable shape parameters

We present an efficient greedy algorithm for constructing sparse radial basis function (RBF) approximations with spatially variable shape parameters. The central idea is to incrementally construct a sparse approximation by greedily selecting a subset of basis functions from a parameterized dictionary consisting of RBFs centered at all of the training points. An incremental thin QR update scheme based on the Gram–Schmidt process with reorthogonalization is employed to efficiently update the weights of the sparse RBF approximation at each iteration. In addition, the shape parameter of the basis function chosen at each iteration is tuned by minimizing the l2-norm of the training residual, while an approximate leave-one-out error metric is used as the dominant stopping criterion. Numerical studies are presented for a range of test functions to demonstrate that the proposed algorithm enables the efficient construction of RBF approximations with spatially variable shape parameters. It is shown that, compared to a classical RBF model with a single tunable shape parameter and Gaussian process models with an anisotropic Gaussian covariance function, the proposed algorithm can provide significant improvements in accuracy, cost, and sparsity, particularly for high-dimensional datasets.