The Probability That a Numerical, Analysis Problem Is Difficult

Numerous problems in numerical analysis, including matrix inversion, eigenvalue calculations and polynomial zerofinding, share the following property: The difficulty of solving a given problem is large when the distance from that problem to the nearest "ill-posed" one is small. For example, the closer a matrix is to the set of noninvertible matrices, the larger its condition number with respect to inversion. We show that the sets of ill-posed problems for matrix inversion, eigenproblems, and polynomial zerofinding all have a common algebraic and geometric structure which lets us compute the probability distribution of the distance from a "random" problem to the set. From this probability distribution we derive, for example, the distribution of the condition number of a random matrix. We examine the relevance of this theory to the analysis and construction of numerical algorithms destined to be run in finite precision arithmetic.

[1]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[2]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[3]  H. Weyl.  On the Volume of Tubes , 1939 .

[4]  H. Hotelling.  Tubes and Spheres in n-Spaces, and a Class of Statistical Problems , 1939 .

[5]  James Hardy Wilkinson,et al.  Rounding errors in algebraic processes , 1964, IFIP Congress.

[6]  J. H. Wilkinson.  The algebraic eigenvalue problem , 1966 .

[7]  W. Kahan.  Numerical Linear Algebra , 1966, Canadian Mathematical Bulletin.

[8]  Tosio Kato.  Perturbation theory for linear operators , 1966 .

[9]  G. Stolzenberg,et al.  Volumes, Limits and Extensions of Analytic Varieties , 1966 .

[10]  J. Rice.  A Theory of Condition , 1966 .

[11]  Paul R. Thie.  The Lelong number of a point of a complex analytic set , 1967 .

[12]  Donald Ervin Knuth,et al.  The Art of Computer Programming , 1968 .

[13]  H. Fédérer.  Geometric Measure Theory , 1969 .

[14]  Axel Ruhe.  Properties of a matrix with a very ill-conditioned eigenproblem , 1970 .

[15]  R. W. Hamming,et al.  On the distribution of numbers , 1970, Bell Syst. Tech. J..

[16]  V. Arnold.  ON MATRICES DEPENDING ON PARAMETERS , 1971 .

[17]  W. Kahan.  Conserving Confluence Curbs Ill-Condition , 1972 .

[18]  J. H. Wilkinson.  Note on matrices with a very ill-conditioned eigenproblem , 1972 .

[19]  G. Stewart.  Error and Perturbation Bounds for Subspaces Associated with Certain Eigenvalue Problems , 1973 .

[20]  Keith Kendig.  Elementary algebraic geometry , 1976 .

[21]  L. Santaló.  Integral geometry and geometric probability , 1976 .

[22]  D. Hough.  Explaining and ameliorating the ill condition of zeros of polynomials. , 1977 .

[23]  P. Griffiths.  Complex differential and integral geometry and curvature integrals associated to singularities of complex analytic varieties , 1978 .

[24]  Donald E. Knuth,et al.  The Art of Computer Programming, Vol. 2 , 1981 .

[25]  S. Smale.  The fundamental theorem of algebra and complexity theory , 1981 .

[26]  A. Gray.  Comparison theorems for the volumes of tubes as generalizations of the Weyl tube formula , 1982 .

[27]  Gene H. Golub,et al.  Matrix computations , 1983 .

[28]  J. H. Wilkinson.  On neighbouring matrices with quadratic elementary divisors , 1984 .

[29]  E. Kostlan.  Statistical complexity of numerical linear algebra , 1985 .

[30]  James Renegar,et al.  On the Efficiency of Newton's Method in Approximating All Zeros of a System of Complex Polynomials , 1987, Math. Oper. Res..

[31]  J. Demmel.  On condition numbers and the distance to the nearest ill-posed problem , 2015 .

[32]  Donald E. Knuth,et al.  The art of computer programming, volume 3: (2nd ed.) sorting and searching , 1998 .

[33]  Steve Smale,et al.  Algorithms for Solving Equations , 2010 .

[34]  Philipp Birken,et al.  Numerical Linear Algebra , 2011, Encyclopedia of Parallel Computing.

[35]  P. Lelong.  Fonctions Plurisousharmoniques Et Formes Differentielles Positives , 2011 .