Approximation in normal linear spaces

Abstract A historical account is given of the development of methods for solving approximation problems set in normed linear spaces. Approximation of both real functions and real data is considered, with particular reference to Lp (or lp) and Chebyshev norms. As well as coverage of methods for the usual linear problems, an account is given of the development of methods for approximation by functions which are nonlinear in the free parameters, and special attention is paid to some particular nonlinear approximating families.

[1]  J. Rice.  The approximation of functions , 1964 .

[2]  L. Schumaker,et al.  Surface Fitting and Multiresolution Methods , 1997 .

[3]  I. Barrodale,et al.  Applications of Mathematical Programming to ℓp Approximation , 1970 .

[4]  An algorithm for chebyshev approximation by rationals with constrained denominators , 1996 .

[5]  J. L. Walsh,et al.  LEAST P-TH POWER POLYNOMIALS ON A REAL FINITE POINT SET , 1955 .

[6]  Elwood T. Olsen,et al.  L1 and L∞ minimization via a variant of Karmarkar's algorithm , 1989, IEEE Trans. Acoust. Speech Signal Process..

[7]  Paul G. Hoel,et al.  Certain problems in the theory of closest approximation , 1935 .

[8]  E. Stiefel.  Note on Jordan elimination, linear programming and Tchebycheff approximation , 1960 .

[9]  Richard Bellman,et al.  Methods of Nonlinear Analysis, Vol. I (Mathematics in Science and Engineering Volume 61-I) , 1973 .

[10]  Nabih N. Abdelmalek,et al.  Linear L1 Approximation for a Discrete Point Set and L1 Solutions of Overdetermined Linear Equations , 1971, J. ACM.

[11]  Kaj Madsen,et al.  A Finite Smoothing Algorithm for Linear l1 Estimation , 1993, SIAM J. Optim..

[12]  Abraham Charnes,et al.  Optimal Estimation of Executive Compensation by Linear Programming , 1955 .

[13]  ON n-PARAMETER FAMILIES OF FUNCTIONS AND ASSOCIATED CONVEX FUNCTIONS , 1950 .

[14]  Sven-Åke Gustafson,et al.  On the Computational Solution of a Class of Generalized Moment Problems , 1970 .

[15]  E. H. Kaufman,et al.  A combined Remes-differential correction algorithm for rational approximation , 1978 .

[16]  P. D Robers,et al.  An interval programming algorithm for discrete linear L1 approximation problems , 1969 .

[17]  T. Coleman,et al.  A global and quadratically convergent method for linear l ∞ problems , 1992 .

[18]  N. Liron,et al.  DCR2: An improved algorithm forl∞ rational approximation on intervals , 1978 .

[19]  R. S. Johnson.  On monosplines of least deviation , 1960 .

[20]  M. Powell,et al.  A Technique that Gains Speed and Accuracy in the Minimax Solution of Overdetermined Linear Equations , 1977 .

[21]  A. Haar,et al.  Die Minkowskische Geometrie und die Annäherung an stetige Funktionen , 1917 .

[22]  T. N. E. Greville,et al.  Theory and applications of spline functions , 1969 .

[23]  Ronald D. Armstrong,et al.  A dual method for discrete Chebychev curve fitting , 1979, ACM '78.

[24]  Ludwig J. Cromme,et al.  Eine Klasse von Verfahren zur Ermittlung bester nichtlinearer Tschebyscheff-Approximationen , 1975 .

[25]  R. Farebrother.  Fitting Linear Relationships: A History of the Calculus of Observations 1750-1900 , 1998 .

[26]  R. Esch,et al.  Computational methods for best spline function approximation , 1969 .

[27]  H. Hurst.  XLVIII. Reducing observations by the method of minimum deviations , 1930 .

[28]  Nabih N. Abdelmalek.  An efficient method for the discrete linear ₁ approximation problem , 1975 .

[29]  M. R. Osborne,et al.  Strong uniqueness and second order convergence in nonlinear discrete approximation , 1980 .

[30]  G. Alistair Watson,et al.  An Algorithm for Minimax Approximation in the Nonlinear Case , 1969, Comput. J..

[31]  H. L. Loeb,et al.  GENERALIZED RATIONAL APPROXIMATION , 1964 .

[32]  S. Bernstein,et al.  Leçons sur les propriétés extrémales et la meilleure approximation des fonctions analytiques d'une variable réelle , 1926 .

[33]  Mustafa Ç. Pinar,et al.  New characterizations of ℓ1 solutions to overdetermined systems of linear equations , 1994, Oper. Res. Lett..

[34]  H. Watanabe,et al.  An Iterative Chebyshev Approximation Method for Network Design , 1968 .

[35]  H. M. Wagner.  Linear Programming Techniques for Regression Analysis , 1959 .

[36]  J. Rice.  The Characterization of Best Nonlinear Tchebycheff Approximations , 1960 .

[37]  J. T. Marti.  A Method for the Numerical Computation of Best L 1 -Approximations of Continuous Functions , 1975 .

[38]  Günther Nürnberger,et al.  A Remez type algorithm for spline functions , 1983 .

[39]  R. Koenker,et al.  The Gaussian hare and the Laplacian tortoise: computability of squared-error versus absolute-error estimators , 1997 .

[40]  Günther Nürnberger,et al.  Algorithms for piecewise polynomials and splines with free knots , 1989 .

[41]  M. J. D. Powell,et al.  The Differential Correction Algorithm for Rational $\ell _\infty $-Approximation , 1972 .

[42]  J. E. Kelley.  An Application of Linear Programming to Curve Fitting , 1958 .

[43]  E. Stiefel,et al.  Über diskrete und lineare Tschebyscheff-Approximationen , 1959, Numerische Mathematik.

[44]  Charles A. Micchelli,et al.  Best L1 approximation by weak chebyshev systems and the uniqueness of interpolating perfect splines , 1977 .

[45]  Narendra Karmarkar,et al.  A new polynomial-time algorithm for linear programming , 1984, Comb..

[46]  C. Dunham.  Approximation with one (or few) parameters nonlinear , 1988 .

[47]  G. Nürnberger,et al.  An algorithm for segment approximation , 1986 .

[48]  L. Schumaker,et al.  Approximation by generalized splines , 1985 .

[49]  Sanjay Mehrotra,et al.  On the Implementation of a Primal-Dual Interior Point Method , 1992, SIAM J. Optim..

[50]  M. Powell,et al.  Approximation theory and methods , 1984 .

[51]  Yuying Li,et al.  A Globally Convergent Method for lp Problems , 1991, SIAM J. Optim..

[52]  L. Cromme.  Strong uniqueness , 1978 .

[53]  Rembert Reemtsen,et al.  Modifications of the first Remez algorithm , 1990 .

[54]  G. Nürnberger.  Approximation by univariate and bivariate splines , 1993 .

[55]  P. T. Breuer.  A new method for real rational uniform approximation , 1987 .

[56]  Kaj Madsen,et al.  Combined LP and Quasi-Newton Methods for Nonlinear $l_1 $ Optimization , 1985 .

[57]  J. Claerbout,et al.  Robust Modeling With Erratic Data , 1973 .

[58]  André L. Tits,et al.  Erratum: An SQP Algorithm for Finely Discretized Continuous Minimax Problems and Other Minimax Problems with Many Objective Functions , 1998, SIAM J. Optim..

[59]  I. Barrodale,et al.  Algorithms for bestL1 andL∞ linear approximations on a discrete set , 1966 .

[60]  Paul L. Butzer,et al.  P. L. Chebyshev (1821-1894) , 1999 .

[61]  Ian Barrodale,et al.  Algorithm 495: Solution of an Overdetermined System of Linear Equations in the Chebychev Norm [F4] , 1975, TOMS.

[62]  John R. Rice,et al.  Tchebycheff approximations by functions unisolvent of variable degree , 1961 .

[63]  E. Cheney,et al.  A Survey of Methods for Rational Approximation, with Particular Reference to a New Method Based on a Forumla of Darboux , 1963 .

[64]  M. R. Osborne,et al.  Discrete, nonlinear approximation problems in polyhedral norms , 1977 .

[65]  C. J. Price,et al.  Numerical experiments in semi-infinite programming , 1996, Comput. Optim. Appl..

[66]  J. Cooper,et al.  Theory of Approximation , 1960, Mathematical Gazette.

[67]  A. Conn,et al.  Minimization Techniques for Piecewise Differentiable Functions: The l_1 Solution to an Overdetermined Linear System , 1978 .

[68]  Ian Barrodale,et al.  Algorithm 478: Solution of an Overdetermined System of Equations in the l1 Norm [F4] , 1974, Commun. ACM.

[69]  J. Tukey,et al.  The Fitting of Power Series, Meaning Polynomials, Illustrated on Band-Spectroscopic Data , 1974 .

[70]  J. Rice,et al.  Approximation from a curve of functions , 1967 .

[71]  M. R. Osborne,et al.  Nonlinear approximation problems in vector norms , 1978 .

[72]  T. Pavlidis,et al.  Uniform piecewise polynomial approximation with variable joints , 1974 .

[73]  P. J. Laurent,et al.  An algorithm of successive minimization in convex programming , 1978 .

[74]  L. B. Smith,et al.  Algorithm 414: Chebyshev approximation of continuous functions by a Chebyshev system of functions , 1967, CACM.

[75]  Thomas F. Coleman,et al.  A globally and quadratically convergent affine scaling method for linearℓ1 problems , 1992, Math. Program..

[76]  G. Alistair Watson,et al.  On an Algorithm for Discrete Nonlinear L1 Approximation , 1971, Comput. J..

[77]  L. Bittner,et al.  Das Austauschverfahren der linearen Tschebyscheff-Approximation bei nicht erfüllter Haarscher Bedingung , 1961 .

[78]  Larry L. Schumaker,et al.  Uniform Approximation by Tchebycheffian Spline Functions , 1968 .

[79]  D. I. Clark.  The Mathematical Structure of Huber’s M-Estimator , 1985 .

[80]  M. Davies.  Linear Approximation Using the Criterion of Least Total Deviations , 1967 .

[81]  C. Dunham.  Chebyshev approximation by exponential-polynomial sums , 1979 .

[82]  J. C. Mairhuber.  ON HAAR'S THEOREM CONCERNING CHEBYCHEV APPROXIMATION PROBLEMS HAVING UNIQUE SOLUTIONS' , 1956 .

[83]  Josef Stoer,et al.  A Direct Method for Chebyshev Approximation by Rational Functions , 1964, JACM.

[84]  T. Motzkin.  Approximation by curves of a unisolvent family , 1949 .

[85]  G. Watson.  Approximation theory and numerical methods , 1980 .

[86]  O. L. Mangasarian,et al.  Linear and Convex Programming , 1966 .

[87]  R. Singleton.  A Method for Minimizing the Sum of Absolute Values of Deviations , 1940 .

[88]  H. L. Loeb,et al.  Further Remarks on the Differential Correction Algorithm , 1973 .

[89]  A Descent Method for the Uniform Solution to Over-Determined Systems of Linear Equations , 1976 .

[90]  John R. Rice,et al.  A moment problem in ₁ approximation , 1965 .

[91]  Rainer Hettich,et al.  An implementation of a discretization method for semi-infinite programming , 1986, Math. Program..

[92]  S. P. Han,et al.  Variable metric methods for minimizing a class of nondifferentiable functions , 1977, Math. Program..

[93]  Kaj Madsen,et al.  A 2-Stage Algorithm for Minimax Optimization , 1979 .

[94]  D. Braess.  Chebyshev approximation by spline functions with free knots , 1971 .

[95]  E. C. Rhodes D.Sc..  LXXXIX. Reducing observations by the method of minimum deviations , 1930 .

[96]  Wu Li,et al.  The Linear l1 Estimator and the Huber M-Estimator , 1998, SIAM J. Optim..

[97]  J. L. Walsh,et al.  Least pth Power Polynomials on a Real Finite Point Set , 1955 .

[98]  Geometrical characterizations for nonlinear uniform approximation , 1974 .

[99]  Michael R. Osborne,et al.  The Construction of Minimax Rational Approximations to Functions , 1966, Comput. J..

[100]  G. A. Watson,et al.  A Multiple Exchange Algorithm for Multivariate Chebyshev Approximation , 1975 .

[101]  Jerry M Wolfe.  On the convergence of an algorithm for discrete L p approximation , 1979 .

[102]  G. Watson,et al.  Numerical Methods for Nonlinear Discrete L1 Approximation Problems , 1980 .

[103]  R. C. James.  Orthogonality and linear functionals in normed linear spaces , 1947 .

[104]  Kristján Jónasson,et al.  Corrected sequential linear programming for sparse minimax optimization , 1994 .

[105]  Hans J. Maehly,et al.  Methods for Fitting Rational Approximations, Parts II and III , 1963, JACM.

[106]  G. Nürnberger.  Strongly unique spline approximations with free knots , 1987 .

[107]  Ronald D. Armstrong,et al.  A dual method for discrete Chebychev curve fitting , 1980, Math. Program..

[108]  R. Fletcher,et al.  Linear Minimax Approximation as the Limit of Best $L_p $-Approximation , 1974 .

[109]  Lecons sur les fonctions de variables reelles , 1905 .

[110]  Friedrich Riesz.  Über lineare Funktionalgleichungen , 1916 .

[111]  Y. Dodge.  on Statistical data analysis based on the L1-norm and related methods , 1987 .

[112]  Paul T. Boggs,et al.  A new algorithm for the Chebyshev solution of overdetermined linear systems , 1974 .

[113]  D. Kammler.  CHEBYSHEV APPROXIMATION OF COMPLETELY MONOTONIC FUNCTIONS BY SUMS OF EXPONENTIALS , 1976 .

[114]  K. Glashoff,et al.  Über die genaue Berechnung von besten L1-Approximierenden , 1979 .

[115]  The weakened first algorithm of Remez , 1981 .

[116]  A. A. Goldstein,et al.  Newton's method for convex programming and Tchebycheff approximation , 1959, Numerische Mathematik.

[117]  Algorithm 458: discrete linear L 1 approximation by interval linear programming , 1973 .

[118]  R. Fletcher.  NUMERICAL EXPERIMENTS WITH AN EXACT L1 PENALTY FUNCTION METHOD , 1981 .

[119]  Die Numerische Berechnung von Startnäherungen bei der Exponentialapproximation , 1978 .

[120]  Aaron Rose.  International symposium on systems optimization and analysis: Bensoussan, A., and Lions, J.L. (Eds), Berlin: Springer-Verlag, 1979 , 1981 .

[121]  L. Schumaker,et al.  Uniform approximation by generalized splines with free knots , 1989 .

[122]  A. Conn,et al.  Primal methods are better than dual methods for solving overdetermined linear systems in the l ∞ sense? , 1989 .

[123]  Anthony Ralston,et al.  Rational chebyshev approximation by Remes' algorithms , 1965 .

[124]  Günther Nürnberger,et al.  Bivariate segment approximation and splines , 1996, Adv. Comput. Math..

[125]  R. Hettich,et al.  A newton-method for nonlinear Chebyshev approximation , 1976 .

[126]  S. Singh,et al.  Approximation theory and spline functions , 1984 .

[127]  Yin Zhang.  Primal-dual interior point approach for computingl1-solutions andl∞-solutions of overdetermined linear systems , 1993 .

[129]  Günther Nürnberger,et al.  Bivariate segment approximation and free knot splines: Research problems 96-4 , 1996 .

[130]  J. Rice.  Chebyshev Approximation by Exponentials , 1962 .

[131]  Rembert Reemtsen,et al.  Numerical Methods for Semi-Infinite Programming: A Survey , 1998 .

[132]  J. Mason,et al.  Algorithms for approximation , 1987 .

[133]  W. Steiger,et al.  Least Absolute Deviations: Theory, Applications and Algorithms , 1984 .

[134]  Richard Bellman,et al.  Methods Of Nonlinear Analysis , 1970 .

[135]  H. Werner.  Der Existenzsatz für das Tschebyscheffsche Approximations-Problem mit Exponentialsummen , 1969 .

[136]  Andrew Young,et al.  Discrete Approximation in the L1 Norm , 1973, Comput. J..

[137]  A. Fam,et al.  An algorithm for computing continuous Chebyshev approximations , 1987 .

[138]  C. Dunham.  Chebyshev approximation by A* ϕ (Bx) , 1973 .

[139]  C. Lawson.  Characteristic properties of the segmented rational minmax approximation problem , 1964 .

[140]  An Algorithm for Discrete, Nonlinear, Best Approximation Problems , 1972 .

[141]  R. Reemtsen,et al.  Semi‐Infinite Programming , 1998 .

[142]  T. Motzkin,et al.  Least th power polynomials on a finite point set , 1956 .

[143]  F. Y. Edgeworth.  XXII. On a new method of reducing observations relating to several quantities , 1888 .

[144]  G. Watson.  The Minimax Solution of an Overdetermined System of Non-linear Equations , 1979 .

[145]  A. Conn,et al.  On Cline’s Direct Method for Solving Overdetermined Linear Systems in the $L_\infty $ Sense , 1978 .

[146]  Gene H. Golub,et al.  Numerical Analysis: Stable numerical methods for obtaining the Chebyshev solution to an overdetermined system of equations , 1968, Commun. ACM.

[147]  Jerry M Wolfe,et al.  On the convergence of an algorithm for discreteLp approximation , 1979 .

[148]  G. D. Taylor,et al.  Uniform Approximation by Rational Functions Having Restricted Denominators. , 1981 .

[149]  G. Alexits.  Approximation theory , 1983 .

[150]  H. L. Loeb,et al.  Two new algorithms for rational approximation , 1961 .

[151]  Helmut Werner,et al.  Rationale tschebyscheff-approximation, eigenwerttheorie und differenzenrechnung , 1963 .

[152]  Karl H. Usow,et al.  On L_1 Approximation II: Computation for Discrete Functions and Discretization Effects , 1967 .

[153]  R. Reemtsen,et al.  Discretization methods for the solution of semi-infinite programming problems , 1991 .

[154]  Paul Kirchberger,et al.  Über Tchebychefsche Annäherungsmethoden , 1903 .

[155]  A. Ralston.  Some Aspects of Degeneracy in Rational Approximations , 1973 .

[156]  I. Singer,et al.  Uniqueness and strong uniqueness of best approximations by spline subspaces and other subspaces , 1982 .

[157]  K. Madsen.  An Algorithm for Minimax Solution of Overdetermined Systems of Non-linear Equations , 1975 .

[158]  H. L. Loeb,et al.  On rational Chebyshev approximation , 1962 .

[159]  A. Kinsella,et al.  Finite Algorithms in Optimisation and Data Analysis. , 1986 .

[160]  G. A. Watson,et al.  Numerical analysis : A.R. Mitchell 75th birthday volume , 1996 .

[161]  W. Fraser,et al.  On the computation of rational approximations to continuous functions , 1962, CACM.

[162]  G. A. Watson,et al.  A Note on Singular Minimax Approximation Problems , 1969 .

[163]  David G. Moursund.  Optimal starting values for Newton-Raphson calculation of x1 2 , 1967, CACM.

[164]  Jürgen Fischer.  The convergence of the best discrete linear Lp approximation as p → 1 , 1983 .

[165]  H. Blatt.  Exchange Algorithms, Error Estimations and Strong Unicity in Convex Programming and Chebyshev Approximation , 1984 .

[166]  L. Rogge,et al.  Natural choice of L1-approximants , 1981 .

[167]  G. A. Watson,et al.  Linear Chebyshev approximation without Chebyshev sets , 1976 .

[168]  A. Conn,et al.  An approach to nonlinear l1 data fitting , 1982 .

[169]  G. Alistair Watson,et al.  On the Best Linear Chebyshev Approximation , 1967, Computer/law journal.

[170]  Helmut Werner,et al.  Die konstruktive Ermittlung der Tschebyscheff-Approximierenden im Bereich der rationalen funktionen , 1962 .

[171]  W. Murray,et al.  A Projected Lagrangian Algorithm for Nonlinear $l_1 $ Optimization , 1981 .

[172]  L. Karlovitz,et al.  Equioscillation under nonuniqueness in the approximation of continuous functions , 1970 .

[173]  W. Murray,et al.  A Projected Lagrangian Algorithm for Nonlinear Minimax Optimization , 1980 .

[174]  I. Barrodale,et al.  An Improved Algorithm for Discrete $l_1 $ Linear Approximation , 1973 .

[175]  M. R. Osborne.  Finite Algorithms in Optimization and Data Analysis , 1985 .

[176]  D. Jackson,et al.  Note on a class of polynomials of approximation , 1921 .

[177]  G. A. Watson.  An Algorithm for Linear L 1 Approximation of Continuous Functions , 1981 .

[178]  Kenneth O. Kortanek,et al.  Numerical treatment of a class of semi‐infinite programming problems , 1973 .

[179]  R. Fletcher,et al.  An algorithm for composite nonsmooth optimization problems , 1986 .

[180]  J. L. Walsh,et al.  The existence of rational functions of best approximation , 1931 .

[181]  L. Veidinger,et al.  On the numerical determination of the best approximations in the Chebyshev sense , 1960 .

[182]  Karl H. Usow,et al.  On $L_1 $ Approximation I: Computation for Continuous Functions and Continuous Dependence , 1967 .

[183]  Günter Meinardus,et al.  Nicht-lineare approximationen , 1964 .

[184]  M. R. Osborne,et al.  The efficient computation of linear rank statistics , 1990 .

[185]  Charles de La Vallee Poussin,et al.  Sur la méthode de l'approximation minimum , 1911 .

[186]  R. Armstrong,et al.  Two linear programming algorithms for the linear discrete ₁ norm problem , 1979 .

[187]  Chebyshev Approximation by $ab^x + c$ , 1960 .

[188]  R. Hettich,et al.  An algorithm for general restricted rational Chebyshev approximation , 1990 .

[189]  The differential correction algorithm for generalized rational functions , 1987 .

[190]  Olvi L. Mangasarian,et al.  Nonlinear Programming , 1969 .

[191]  John R. Rice,et al.  Characterization of Chebyshev Approximations by Splines , 1967 .

[192]  An Efficient Algorithm for Nonlinear Minimax Problems , 1990 .

[193]  John Wesley Young.  General theory of approximation by functions involving a given number of arbitrary parameters , 1907 .

[194]  Walter D. Fisher.  A Note on Curve Fitting with Minimum Deviations by Linear Programming , 1961 .

[195]  G. Watson.  A Method for Calculating Best Non-linear Chebyshev Approximations , 1976 .

[196]  Jean Descloux,et al.  Approximations in $L^p $ and Chebyshev Approximations , 1963 .

[197]  Andrew Young,et al.  A Note on Numerical Procedures for Approximation by Spline Functions , 1966, Comput. J..