On the Power of Adaption

Abstract Optimal error bounds for adaptive and nonadaptive numerical methods are compared. Since the class of adaptive methods is much larger, a well-chosen adaptive method might seem to be better than any nonadaptive method. Nevertheless there are several results saying that under natural assumptions adaptive methods are not better than nonadaptive ones. There are also other results, however, saying that adaptive methods can be significantly better than nonadaptive ones as well as bounds on how much better they can be. It turns out that the answer to the “adaption problem” depends very much on what is known a priori about the problem in question; even a seemingly small change of the assumptions can lead to a different answer.

[1]  Klaus Ritter,et al.  Average errors for zero finding: Lower bounds for smooth or monotone functions , 1994 .

[2]  Erich Novak,et al.  Quadrature Formulas for Multivariate Convex Functions , 1996, J. Complex..

[3]  Ming Chu,et al.  There Exists a Problem Whose Computational Complexity Is Any Given Function of the Information Complexity , 1994, J. Complex..

[4]  J. Calvin.  Average performance of a class of adaptive algorithms for global optimization , 1997 .

[5]  K. Ritter,et al.  High dimensional integration of smooth functions over cubes , 1996 .

[6]  Stamatis Cambanis,et al.  13 Sampling designs for time series , 1985 .

[7]  J. F. Traub,et al.  Information-Based complexity: New questions for mathematicians , 1991 .

[8]  H. Wozniakowski.  Average case complexity of linear multivariate problems , 1993, math/9307234.

[9]  Klaus Ritter,et al.  Approximation and optimization on the Wiener space , 1990, J. Complex..

[10]  J. Bowen,et al.  s -numbers in information-based complexity , 1990 .

[11]  M. Kon,et al.  The adaption problem for approximating linear operators , 1990 .

[12]  R. P. Fedorenko.  The speed of convergence of one iterative process , 1964 .

[13]  G. Wahba.  Spline models for observational data , 1990 .

[14]  G. Wasilkowski.  Integration and approximation of multivariate functions: average case complexity with isotropic Wiener measure , 1993, math/9304216.

[15]  I. Sloan.  Lattice Methods for Multiple Integration , 1994 .

[16]  Spassimir H. Paskov,et al.  Average Case Complexity of Multivariate Integration for Smooth Functions , 1993, J. Complex..

[17]  N. S. Bakhvalov,et al.  On the optimality of linear methods for operator approximation in convex classes of functions , 1971 .

[18]  J. Mockus.  Bayesian Approach to Global Optimization: Theory and Applications , 1989 .

[19]  Andreas Griewank,et al.  Numerical experiments with partially separable optimization problems , 1984 .

[20]  Christopher A. Sikorski.  Optimal solution of nonlinear equations , 1985, J. Complex..

[21]  Seymour Haber,et al.  Numerical Evaluation of Multiple Integrals , 1970 .

[22]  Aleksei G. Sukharev,et al.  On adaptive and nonadaptive stochastic and deterministic algorithms , 1990, J. Complex..

[23]  Grzegorz W. Wasilkowski,et al.  On average complexity of global optimization problems , 1992, Math. Program..

[24]  Michael Griebel,et al.  A combination technique for the solution of sparse grid problems , 1990, Forschungsberichte, TU Munich.

[25]  G. Sonnevend.  An optimal sequential algorithm for the uniform approximation of convex functions on $[0,1]^2$. , 1985 .

[26]  Leszek Plaskota.  A Note on Varying Cardinality in the Average Case Setting , 1993, J. Complex..

[27]  Stefan Heinrich,et al.  The Monte Carlo complexity of Fredholm integral equations , 1993 .

[28]  Boleslaw Z. Kacewicz.  On sequential and parallel solution of initial value problems , 1990, J. Complex..

[29]  Edward W. Packel.  Do Linear Problems Have Linear Optimal Algorithms , 1988 .

[30]  E. Novak.  The Adaption Problem for Nonsymmetric Convex Sets , 1995 .

[31]  Arthur G. Werschulz.  The Complexity of Two-Point Boundary-Value Problems with Piecewise Analytic Data , 1994, J. Complex..

[32]  Wolfgang Hackbusch,et al.  Multi-grid methods and applications , 1985, Springer series in computational mathematics.

[33]  Erich Novak,et al.  On the adaptive and continuous information problems , 1989, J. Complex..

[34]  J. Baumeister.  Stable solution of inverse problems , 1987 .

[35]  G. Wahba,et al.  Hybrid Adaptive Splines , 1997 .

[36]  Erich Novak,et al.  Average errors for zero finding: Lower bounds , 1992 .

[37]  R. Gorenflo,et al.  Abel Integral Equations: Analysis and Applications , 1991 .

[38]  Y. Vardi,et al.  From image deblurring to optimal investments : maximum likelihood solutions for positive linear inverse problems , 1993 .

[39]  Grzegorz W. Wasilkowski.  Some nonlinear problems are as easy as the approximation problem , 1984 .

[40]  E. Novak.  Quadrature formulas for monotone functions , 1992 .

[41]  Aleksei G. Sukharev.  The concept of sequential optimality for problems in numerical analysis , 1987, J. Complex..

[42]  A. Pietsch.  Eigenvalues and S-Numbers , 1987 .

[43]  N. Bakhvalov.  On the convergence of a relaxation method with natural constraints on the elliptic operator , 1966 .

[44]  B. Kacewicz.  How to increase the order to get minimal-error algorithms for systems of ODE , 1984 .

[45]  Arthur G. Werschulz,et al.  Computational complexity of differential and integral equations - an information-based approach , 1991, Oxford mathematical monographs.

[46]  Martin D. Buhmann,et al.  Boolean methods in interpolation and approximation , 1990, Acta Applicandae Mathematicae.

[47]  I. Johnstone,et al.  Wavelet Shrinkage: Asymptopia? , 1995 .

[48]  G. Sonnevend,et al.  An optimal sequential algorithm for the uniform approximation of convex functions on [0,1]2 , 1983 .

[49]  Henryk Wozniakowski,et al.  There Exists a Linear Problem with Infinite Combinatory Complexity , 1993, J. Complex..

[50]  Grzegorz W. Wasilkowski,et al.  The Computational Complexity of Differential and Integral Equations: An Information-Based Approach (Arthur G. Werschulz) , 1994, SIAM Rev..

[51]  P. Hansen.  Numerical tools for analysis and solution of Fredholm integral equations of the first kind , 1992 .

[52]  H. Woźniakowski,et al.  Can adaption help on the average? , 1984 .

[53]  A. Pinkus.  n-Widths in Approximation Theory , 1985 .

[54]  Erich Novak,et al.  Optimal stochastic quadrature formulas for convex functions , 1994 .

[55]  Aimo A. Törn,et al.  Global Optimization , 1999, Science.

[56]  C. Micchelli,et al.  Optimal Sequential and Non-Sequential Procedures for Evaluating a Functional. , 1980 .

[57]  S. Smale,et al.  On a theory of computation and complexity over the real numbers; np-completeness , 1989 .

[58]  W. Rheinboldt,et al.  Error Estimates for Adaptive Finite Element Computations , 1978 .

[59]  Nikolaj P. Korneichuk.  Optimization of Active Algorithms for Recovery of Monotonic Functions from Hölder's Class , 1994, J. Complex..

[60]  Erich Novak,et al.  The Real Number Model in Numerical Analysis , 1995, J. Complex..

[61]  Dany Leviatan,et al.  Wavelet compression and nonlinearn-widths , 1993, Adv. Comput. Math..

[62]  Katrina Bogues,et al.  An implementation of the method of Ermakov and Zolotukhin for multidimensional integration and interpolation , 1981 .

[63]  Grzegorz W. Wasilkowski,et al.  Information of varying cardinality , 1986, J. Complex..

[64]  Erich Novak,et al.  Some Complexity Results for Zero Finding for Univariate Functions , 1993, J. Complex..

[65]  J. Kiefer.  Optimum Sequential Search and Approximation Methods Under Minimum Regularity Assumptions , 1957 .

[66]  C. W. Groetsch,et al.  Inverse Problems in the Mathematical Sciences , 1993 .

[67]  Leszek Plaskota,et al.  Worst Case Complexity of Problems with Random Information Noise , 1996, J. Complex..

[68]  John Darzentas,et al.  Problem Complexity and Method Efficiency in Optimization , 1983 .

[69]  Henryk Wozniakowski,et al.  A general theory of optimal algorithms , 1980, ACM monograph series.

[70]  Erich Novak,et al.  The average error of quadrature formulas for functions of bounded variation , 1990 .

[71]  Erich Novak,et al.  Average-case results for zero finding , 1989, J. Complex..

[72]  Nicholas I. M. Gould,et al.  Lancelot: A FORTRAN Package for Large-Scale Nonlinear Optimization (Release A) , 1992 .

[73]  Erich Novak.  Stochastic properties of quadrature formulas , 1988 .

[74]  E. Novak.  Quadrature Formulas For Convex Classes of Functions , 1993 .

[75]  R. DeVore,et al.  Optimal nonlinear approximation , 1989 .

[76]  George G. Lorentz,et al.  Constructive Approximation , 1993, Grundlehren der mathematischen Wissenschaften.

[77]  V. N. Temli︠a︡kov.  Approximation of functions with bounded mixed derivative , 1989 .

[78]  Alan Genz,et al.  An adaptive algorithm for numerical integration over an n-dimensional rectangular region , 1980 .

[79]  J. Traub.  Iterative Methods for the Solution of Equations , 1982 .

[80]  Vladimir N. Temlyakov,et al.  On Approximate Recovery of Functions with Bounded Mixed Derivative , 1993, J. Complex..

[81]  Andrew F. Siegel,et al.  Unbiased Monte Carlo Integration Methods with Exactness for Low Order Polynomials , 1985 .

[82]  I. P. Huerta.  Adaption helps for some nonconvex classes , 1986, J. Complex..

[83]  Erich Novak,et al.  Global Optimization Using Hyperbolic Cross Points , 1996 .

[84]  Grzegorz W. Wasilkowski,et al.  Randomization for continuous problems , 1989, J. Complex..

[85]  Leszek Plaskota,et al.  Noisy information and computational complexity , 1996 .

[86]  D. Hào,et al.  A mollification method for ill-posed problems , 1994 .

[87]  K. Ritter,et al.  MULTIVARIATE INTEGRATION AND APPROXIMATION FOR RANDOM FIELDS SATISFYING SACKS-YLVISAKER CONDITIONS , 1995 .

[88]  K. Ritter.  Asymptotic optimality of regular sequence designs , 1996 .

[89]  Erich Novak,et al.  Average-case optimality of a hybrid secant-bisection method , 1995 .

[90]  Stephen Smale,et al.  Some Remarks on the Foundations of Numerical Analysis , 1990, SIAM Rev..

[91]  Anargyros Papageorgiou.  Integration of Monotone Functions of Several Variables , 1993, J. Complex..

[92]  T. Müller-Gronbach.  Optimal designs for approximating the path of a stochastic process , 1996 .

[93]  Erich Novak,et al.  Numerical Integration of Peak Functions , 1996, J. Complex..

[94]  Daniel Zwillinger,et al.  Handbook of Integration. , 1992 .

[95]  Henryk Wozniakowski,et al.  Explicit Cost Bounds of Algorithms for Multivariate Tensor Product Problems , 1995, J. Complex..

[96]  Claes Johnson,et al.  Introduction to Adaptive Methods for Differential Equations , 1995, Acta Numerica.

[97]  E. Novak.  Deterministic and Stochastic Error Bounds in Numerical Analysis , 1988 .

[98]  Vladimir Temlyakov,et al.  APPROXIMATE RECOVERY OF PERIODIC FUNCTIONS OF SEVERAL VARIABLES , 1987 .

[99]  Hubert Schwetlick,et al.  Spline smoothing under constraints on derivatives , 1993 .

[100]  Y. Brudnyi,et al.  Adaptive Approximation of Functions with Singularities , 1990 .

[101]  M. Birman,et al.  PIECEWISE-POLYNOMIAL APPROXIMATIONS OF FUNCTIONS OF THE CLASSES $ W_{p}^{\alpha}$ , 1967 .

[102]  Florian A. Potra,et al.  On enclosing simple roots of nonlinear equations , 1993 .

[103]  G W Wasilowski,et al.  Information of varying cardinality , 1986 .

[104]  Peter Mathé.  s-Numbers in information-based complexity , 1990, J. Complex..

[105]  J. Mockus,et al.  The Bayesian approach to global optimization , 1989 .

[106]  A. G. Zhilinskas,et al.  Single-step Bayesian search method for an extremum of functions of a single variable , 1975 .

[107]  I. Csiszár.  Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems , 1991 .

[108]  Henryk Wozniakowski,et al.  Average case complexity of linear multivariate problems I. Theory , 1992, J. Complex..

[109]  D. Braess.  Nonlinear Approximation Theory , 1986 .

[110]  Henryk Wozniakowski,et al.  Information-based complexity , 1987, Nature.

[111]  Erich Novak,et al.  Bisection is not optimal on the average , 1989 .

[112]  Erich Novak,et al.  Optimal Recovery and n-Widths for Convex Classes of Functions , 1995 .

[113]  Jerome Spanier,et al.  Quasi-Random Methods for Estimating Integrals Using Relatively Small Samples , 1994, SIAM Rev..

[114]  S. Heinrich.  Random Approximation in Numerical Analysis , 1994 .

[115]  David L. Donoho,et al.  Nonlinear Wavelet Methods for Recovery of Signals, Densities, and Spectra from Indirect and Noisy Da , 1993 .

[116]  Grzegorz W. Wasilkowski,et al.  On the power of adaptive information for functions with singularities , 1992 .