Nonlinear approximation

This is a survey of nonlinear approximation, especially that part of the subject which is important in numerical computation. Nonlinear approximation means that the approximants do not come from linear spaces but rather from nonlinear manifolds. The central question to be studied is what, if any, are the advantages of nonlinear approximation over the simpler, more established, linear methods. This question is answered by studying the rate of approximation which is the decrease in error versus the number of parameters in the approximant. The number of parameters usually correlates well with computational effort. It is shown that in many settings the rate of nonlinear approximation can be characterized by certain smoothness conditions which are significantly weaker than required in the linear theory. Emphasis in the survey will be placed on approximation by piecewise polynomials and wavelets as well as their numerical implementation. Results on highly nonlinear methods such as optimal basis selection and greedy algorithms (adaptive pursuit) are also given. Applications to image processing, statistical estimation, regularity for PDEs, and adaptive algorithms are discussed.

[1]  E. Schmidt.  Zur Theorie der linearen und nichtlinearen Integralgleichungen , 1907 .

[2]  A. Calderón.  Intermediate spaces and interpolation, the complex method , 1964 .

[3]  E. Stein.  Singular Integrals and Di?erentiability Properties of Functions , 1971 .

[4]  J. Cooper.  SINGULAR INTEGRALS AND DIFFERENTIABILITY PROPERTIES OF FUNCTIONS , 1973 .

[5]  G. A. Baker.  Essentials of Padé approximants , 1975 .

[6]  H. Johnen,et al.  On the equivalence of the K-functional and moduli of continuity and some applications , 1976, Constructive Theory of Functions of Several Variables.

[7]  POLYGONAL APPROXIMATION OF FUNCTIONS OF TWO VARIABLES , 1979 .

[8]  D. Wulbert.  The Rational Approximation of Real Functions , 1978 .

[9]  G. Pisier.  Remarques sur un résultat non publié de B. Maurey , 1981 .

[10]  O. Oleinik,et al.  Boundary-value problems for partial differential equations in non-smooth domains , 1983 .

[11]  Rong-Qing Jia,et al.  Approximation by Smooth Bivariate Splines on a Three-Direction Mesh. , 1983 .

[12]  Relations between rational and spline approximations , 1984 .

[13]  Ronald A. DeVore,et al.  Multivariate rational approximation , 1986 .

[14]  Henryk Wozniakowski,et al.  Information-based complexity , 1987, Nature.

[15]  C. Bennett,et al.  Interpolation of operators , 1987 .

[16]  Relations between rational and spline approximations in L p metric , 1987 .

[17]  R. DeVore,et al.  Free multivariate splines , 1987 .

[18]  R. DeVore,et al.  Interpolation of Besov-Spaces , 1988 .

[19]  I. Daubechies.  Orthonormal bases of compactly supported wavelets , 1988 .

[20]  P. Petrushev.  Direct and converse theorems for spline and rational approximation and besov spaces , 1988 .

[21]  B. Lucier.  REGULARITY THROUGH APPROXIMATION FOR SCALAR CONSERVATION LAWS , 1988 .

[22]  R. DeVore,et al.  Interpolation spaces and non-linear approximation , 1988 .

[23]  I. J. Schoenberg.  Contributions to the Problem of Approximation of Equidistant Data by Analytic Functions , 1988 .

[24]  R. DeVore,et al.  Optimal nonlinear approximation , 1989 .

[25]  R. DeVore,et al.  Degree of Adaptive Approximation , 1990 .

[26]  P. Oswald.  On the degree of nonlinear spline approximation in Besov-Sobolev spaces , 1990 .

[27]  Ronald A. DeVore,et al.  High order regularity for conservation laws , 1990 .

[28]  Ronald R. Coifman,et al.  Entropy-based algorithms for best basis selection , 1992, IEEE Trans. Inf. Theory.

[29]  L. Jones.  A Simple Lemma on Greedy Approximation in Hilbert Space and Convergence Rates for Projection Pursuit Regression and Neural Network Training , 1992 .

[30]  R. DeVore,et al.  Compression of wavelet decompositions , 1992 .

[31]  I. Daubechies,et al.  Biorthogonal bases of compactly supported wavelets , 1992 .

[32]  Jerome M. Shapiro,et al.  An embedded hierarchical image coder using zerotrees of wavelet coefficients , 1993, [Proceedings] DCC `93: Data Compression Conference.

[33]  Dany Leviatan,et al.  Wavelet compression and nonlinearn-widths , 1993, Adv. Comput. Math..

[34]  I. Daubechies,et al.  Wavelets on the Interval and Fast Wavelet Transforms , 1993 .

[35]  R. DeVore,et al.  BESOV SPACES ON DOMAINS IN Rd , 1993 .

[36]  George G. Lorentz,et al.  Constructive Approximation , 1993, Grundlehren der mathematischen Wissenschaften.

[37]  D. L. Donoho,et al.  Ideal spacial adaptation via wavelet shrinkage , 1994 .

[38]  A. Harten.  Adaptive Multiresolution Schemes for Shock Computations , 1994 .

[39]  A. Cohen.  Ten Lectures on Wavelets, CBMS-NSF Regional Conference Series in Applied Mathematics, Vol. 61, I. Daubechies, SIAM, 1992, xix + 357 pp. , 1994 .

[40]  M. Victor Wickerhauser,et al.  Adapted wavelet analysis from theory to software , 1994 .

[41]  Ivo Babuska,et al.  The p and h-p Versions of the Finite Element Method, Basic Principles and Properties , 1994, SIAM Rev..

[42]  G. Kyriazis.  Approximation from shift-invariant spaces , 1995 .

[43]  Carlos E. Kenig,et al.  The Inhomogeneous Dirichlet Problem in Lipschitz Domains , 1995 .

[44]  Ronald A. DeVore,et al.  On the size and smoothness of solutions to nonlinear hyperbolic conservation laws , 1996 .

[45]  Ronald A. DeVore,et al.  Some remarks on greedy algorithms , 1996, Adv. Comput. Math..

[46]  G. Lorentz,et al.  Constructive approximation : advanced problems , 1996 .

[47]  W. Dahmen.  Wavelet and multiscale methods for operator equations , 1997, Acta Numerica.

[48]  Lawrence Carin,et al.  Matching pursuits with a wave-based dictionary , 1997, IEEE Trans. Signal Process..

[49]  D. Donoho.  CART AND BEST-ORTHO-BASIS: A CONNECTION' , 1997 .

[50]  Michael T. Orchard,et al.  Space-frequency quantization for wavelet image coding , 1997, IEEE Trans. Image Process..

[51]  Ronald A. DeVore,et al.  Using nonlinear wavelet compression to enhance image registration , 1997, Defense, Security, and Sensing.

[52]  S. Mallat,et al.  Adaptive greedy approximations , 1997 .

[53]  R. DeVore,et al.  Besov regularity for elliptic boundary value problems , 1997 .

[54]  Wolfgang Dahmen,et al.  Nonlinear Approximation and Adaptive Techniques for Solving Elliptic Operator Equations , 1997 .

[55]  Antonin Chambolle,et al.  Nonlinear wavelet image processing: variational problems, compression, and noise removal through wavelet shrinkage , 1998, IEEE Trans. Image Process..

[56]  Vladimir N. Temlyakov,et al.  The best m-term approximation and greedy algorithms , 1998, Adv. Comput. Math..

[57]  R. DeVore,et al.  Hyperbolic Wavelet Approximation , 1998 .

[58]  Stéphane Mallat,et al.  Analysis of low bit rate image transform coding , 1998, IEEE Trans. Signal Process..

[59]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[60]  Stephan Dahlke,et al.  Besov regularity for elliptic boundary value problems in polygonal domains , 1999 .

[61]  Joel Ratsaby,et al.  On the Degree of Approximation by Manifolds of Finite Pseudo-Dimension , 1999 .

[62]  R. DeVore,et al.  Nonlinear Approximation and the Space BV(R2) , 1999 .

[63]  R. DeVore,et al.  Nonlinear approximation and the space BV[inline-graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="01i" /] , 1999 .