An alternating variable method for the maximal correlation problem

The maximal correlation problem (MCP) aiming at optimizing correlations between sets of variables plays an important role in many areas of statistical applications. Up to date, algorithms for the general MCP stop at solutions of the multivariate eigenvalue problem (MEP), which serves only as a necessary condition for the global maxima of the MCP. For statistical applications, the global maximizer is quite desirable. In searching the global solution of the MCP, in this paper, we propose an alternating variable method (AVM), which contains a core engine in seeking a global maximizer. We prove that (i) the algorithm converges globally and monotonically to a solution of the MEP, (ii) any convergent point satisfies a global optimal condition of the MCP, and (iii) whenever the involved matrix A is nonnegative irreducible, it converges globally to the global maximizer. These properties imply that the AVM is an effective approach to obtain a global maximizer of the MCP. Numerical testings are carried out and suggest a superior performance to the others, especially in finding a global solution of the MCP.

[1]  H. Hotelling.  Relations Between Two Sets of Variates , 1936 .

[2]  Li-Zhi Liao,et al.  Towards the global solution of the maximal correlation problem , 2011, J. Glob. Optim..

[3]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[4]  José Mario Martínez,et al.  Local Minimizers of Quadratic Functions on Euclidean Balls and Spheres , 1994, SIAM J. Optim..

[5]  Philip E. Gill,et al.  Practical optimization , 1981 .

[6]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[7]  Kim-Chuan Toh,et al.  SDPT3 -- A Matlab Software Package for Semidefinite Programming , 1996 .

[8]  Moody T. Chu,et al.  On a Multivariate Eigenvalue Problem, Part I: Algebraic Theory and a Power Method , 1993, SIAM J. Sci. Comput..

[9]  Jorge J. Moré,et al.  Computing a Trust Region Step , 1983 .

[10]  H. Hotelling.  The most predictable criterion. , 1935 .

[11]  Moody T. Chu,et al.  Computing absolute maximum correlation , 2012 .

[12]  Alberto Seeger,et al.  Spectral analysis of coupled linear complementarity problems , 2010 .

[13]  Paul Horst,et al.  Relations amongm sets of measures , 1961 .

[14]  Patrick J. Browne,et al.  A Numerical Technique for Multiparameter Eigenvalue Problems , 1982 .

[15]  Paul Horst,et al.  Factor analysis of data matrices , 1965 .

[16]  Mohamed Hanafi,et al.  Global optimality of the successive Maxbet algorithm , 2003 .

[17]  J. Berge,et al.  Generalized approaches to the maxbet problem and the maxdiff problem, with applications to canonical correlations , 1988 .

[18]  Y. Nesterov.  Quality of semidefinite relaxation for nonconvex quadratic optimization , 1997 .

[19]  John Shawe-Taylor,et al.  Canonical Correlation Analysis: An Overview with Application to Learning Methods , 2004, Neural Computation.

[20]  J. Geer.  Linear relations amongk sets of variables , 1984 .

[21]  J. Kettenring,et al.  Canonical Analysis of Several Sets of Variables , 2022 .

[22]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[23]  Kim-Chuan Toh,et al.  Solving semidefinite-quadratic-linear programs using SDPT3 , 2003, Math. Program..

[24]  Gene H. Golub,et al.  Continuous methods for extreme and interior eigenvalue problems , 2006 .