Calculating the singular values and pseudo-inverse of a matrix

A numerically stable and fairly fast scheme is described to compute the unitary matrices U and V which transform a given matrix A into a diagonal form $\Sigma = U^ * AV$, thus exhibiting A’s singular values on $\Sigma $’s diagonal. The scheme first transforms A to a bidiagonal matrix J, then diagonalizes J. The scheme described here is complicated but does not suffer from the computational difficulties which occasionally afflict some previously known methods. Some applications are mentioned, in particular the use of the pseudo-inverse $A^I = V\Sigma ^I U^* $ to solve least squares problems in a way which dampens spurious oscillation and cancellation.

[1]  M. Hestenes.  Inversion of Matrices by Biorthogonalization and Related Results , 1958 .

[2]  E. E. Osborne.  On Least Squares Solutions of Linear Equations , 1961, JACM.

[3]  J. H. Wilkinson.  Error Analysis of Eigenvalue Techniques Based on Orthogonal Transformations , 1962 .

[4]  G. Birkhoff,et al.  A survey of modern algebra , 1942 .

[5]  G. Forsythe,et al.  On best conditioned matrices , 1955 .

[6]  D. C. Murdoch,et al.  Elements of Linear Spaces. , 1961 .

[7]  Gene H. Golub,et al.  COMPARISON OF THE VARIANCE OF MINIMUM VARIANCE AND WEIGHTED LEAST SQUARES REGRESSION COEFFICIENTS , 1963 .

[8]  V. Kublanovskaya.  On some algorithms for the solution of the complete eigenvalue problem , 1962 .

[9]  James M. Ortega,et al.  The LLT and QR methods for symmetric tridiagonal matrices , 1963, Comput. J..

[10]  T. Greville,et al.  Some Applications of the Pseudoinverse of a Matrix , 1960 .

[11]  J. H. Wilkinson.  Calculation of the eigenvalues of a symmetric tridiagonal matrix by the method of bisection , 1962 .

[12]  F. L. Bauer.  Optimally scaled matrices , 1963 .

[13]  Garrett Birkhoff,et al.  A survey of modern algebra , 1942 .

[14]  H. Rutishauser.  On jacobi rotation patterns , 1963 .

[15]  W. Givens.  Numerical Computation of the Characteristic Values of a Real Symmetric Matrix , 1954 .

[16]  D. R. Fulkerson,et al.  An algorithm for scaling matrices. , 1962 .

[17]  James Hardy Wilkinson,et al.  Error Analysis of Direct Methods of Matrix Inversion , 1961, JACM.

[18]  David D. Morrison.  Remarks on the Unitary Triangularization of a Nonsymmetric Matrix , 1960, JACM.

[19]  Herman H. Goldstine,et al.  The Jacobi Method for Real Symmetric Matrices , 1959, JACM.

[20]  R. Penrose.  On best approximate solutions of linear matrix equations , 1956, Mathematical Proceedings of the Cambridge Philosophical Society.

[21]  Adi Ben-Israel,et al.  An Elimination Method for Computing the Generalized Inverse of an Arbitrary Complex Matrix , 1963, JACM.

[22]  James Hardy Wilkinson.  Stability of the Reduction of a Matrix to Almost Triangular and Triangular Forms by Elementary Similarity Transformations , 1959, JACM.

[23]  V. Hutson.  Integral Equations , 1967, Nature.

[24]  James Hardy Wilkinson,et al.  Householder's Method for the Solution of the Algebraic Eigenproblem , 1960, Comput. J..

[25]  J. Gillis,et al.  Linear Differential Operators , 1963 .

[26]  J. Boot,et al.  The Computation of the Generalized Inverse of singular or Rectangular Matrices , 1963 .

[27]  Alston S. Householder,et al.  Unitary Triangularization of a Nonsymmetric Matrix , 1958, JACM.

[28]  E. G. Kogbetliantz.  Solution of linear equations by diagonalization of coefficients matrix , 1955 .

[29]  James Hardy Wilkinson,et al.  The Calculation of the Eigenvectors of Codiagonal Matrices , 1958, Comput. J..

[30]  G. Forsythe,et al.  The cyclic Jacobi method for computing the principal values of a complex matrix , 1960 .

[31]  Heinz Rutishauser,et al.  Deflation bei Bandmatrizen , 1959 .

[32]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[33]  A. Charnes,et al.  Contributions to the Theory of Generalized Inverses , 1963 .

[34]  E. E. Osborne.  On pre-conditioning matrices , 1959, ACM '59.