Quadratic minimisation problems in statistics

We consider the problem min"x(x-t)^'A(x-t) subject to x^'Bx+2b^'x=k where A is positive definite or positive semi-definite. Variants of this problem are discussed within the framework of a general unifying methodology. These include non-trivial considerations that arise when (i) A and/or B are not of full rank and (ii) t takes special forms (especially t=0 which, under further conditions, reduces to the well-known two-sided eigenvalue solution). Special emphasis is placed on insights provided by geometrical interpretations.

[1]  W. Gander.  Least squares with a quadratic constraint , 1980 .

[2]  Michael R. Veall,et al.  Formulating Wald Tests of Nonlinear Restrictions , 1985 .

[3]  Jos M. F. ten Berge.  A generalization of Verhelst's solution for a constrained regression problem in alscal and related MDS-algorithms , 1983 .

[4]  J. J. Moré.  Generalizations of the trust region problem , 1993 .

[5]  Michael W. Browne,et al.  On oblique procrustes rotation , 1967 .

[6]  C. Reinsch.  Smoothing by spline functions , 1967 .

[7]  Elliot M. Cramer,et al.  On browne's solution for oblique procrustes rotation , 1974 .

[8]  Henk A. L. Kiers.  Maximization of sums of quotients of quadratic forms and some generalizations , 1995 .

[9]  Shizuhiko Nishisato,et al.  Some results on proper eigenvalues and eigenvectors with applications to scaling , 1979 .

[10]  Forrest W. Young,et al.  Nonmetric individual differences multidimensional scaling: An alternating least squares method with optimal scaling features , 1977 .

[11]  J. Berge,et al.  A general solution to Mosier's oblique procrustes problem , 1977 .

[12]  B. Silverman,et al.  Nonparametric Regression and Generalized Linear Models: A roughness penalty approach , 1993 .

[13]  Hoang Tuy,et al.  A robust algorithm for quadratic optimization under quadratic constraints , 2007, J. Glob. Optim..

[14]  J C Gower.  The role of constraints in determining optimal scores. , 1998, Statistics in medicine.

[15]  Jean-Paul Chilès,et al.  Wiley Series in Probability and Statistics , 2012 .

[16]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[17]  Douglas C. Montgomery,et al.  Response Surface Methodology: Process and Product Optimization Using Designed Experiments , 1995 .

[18]  K. S. Banerjee.  Generalized Inverse of Matrices and Its Applications , 1973 .

[19]  Snigdhansu Chatterjee,et al.  Procrustes Problems , 2005, Technometrics.

[20]  W. Marsden.  I and J , 2012 .

[21]  Paul Marriott,et al.  On the Differential Geometry of the Wald Test With Nonlinear Restrictions , 1996 .

[22]  Lars Eldèn.  Solving Quadratically Constrained Least Squares Problems Using a Differential-Geometric Approach , 2002 .

[23]  B. Silverman,et al.  Nonparametric Regression and Generalized Linear Models: A roughness penalty approach , 1993 .

[24]  M. J. R. Healy,et al.  An approach to the scaling of categorized attributes , 1976 .

[25]  Alan J. Mayne,et al.  Generalized Inverse of Matrices and its Applications , 1972 .

[26]  Kenneth J. White,et al.  Obtaining any Wald statistic you want , 1986 .

[27]  Jan de Leeuw,et al.  Generalized eigenvalue problems with positive semi-definite matrices , 1982 .

[28]  Casper J. Albers,et al.  Applications of quadratic minimisation problems in statistics , 2011, J. Multivar. Anal..

[29]  Susan A. Murphy,et al.  Monographs on statistics and applied probability , 1990 .

[30]  F. Uhlig.  A recurring theorem about pairs of quadratic forms and extensions: a survey , 1979 .

[31]  W. V. McCarthy,et al.  Discriminant Analysis with Singular Covariance Matrices: Methods and Applications to Spectroscopic Data , 1995 .

[32]  Robert W. Newcomb,et al.  On the simultaneous diagonalization of two semi-definite matrices , 1961 .