Towards the global solution of the maximal correlation problem

The maximal correlation problem (MCP) aiming at optimizing correlation between sets of variables plays a very important role in many areas of statistical applications. Currently, algorithms for the general MCP stop at solutions of the multivariate eigenvalue problem for a related matrix A, which serves as a necessary condition for the global solutions of the MCP. However, the reliability of the statistical prediction in applications relies greatly on the global maximizer of the MCP, and would be significantly impacted if the solution found is a local maximizer. Towards the global solution of the MCP, we have obtained four results in the present paper. First, the sufficient and necessary condition for global optimality of the MCP when A is a positive matrix is extended to the nonnegative case. Secondly, the uniqueness of the multivariate eigenvalues in the global maxima of the MCP is proved either when there are only two sets of variables involved, or when A is nonnegative. The uniqueness of the global maximizer of the MCP for the nonnegative irreducible case is also proved. These theoretical achievements lead to our third result that if A is a nonnegative irreducible matrix, both the Horst-Jacobi algorithm and the Gauss-Seidel algorithm converge globally to the global maximizer of the MCP. Lastly, some new estimates of the multivariate eigenvalues related to the global maxima are obtained.

[1]  M. Chu,et al.  On a Multivariate Eigenvalue Problem � I Algebraic Theory and a Power Method , 2004 .

[2]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[3]  J. Geer.  Linear relations amongk sets of variables , 1984 .

[4]  Gene H. Golub,et al.  Continuous methods for extreme and interior eigenvalue problems , 2006 .

[5]  Moody T. Chu,et al.  Computing absolute maximum correlation , 2012 .

[6]  Paul Horst,et al.  Factor analysis of data matrices , 1965 .

[7]  C. Sigg,et al.  Nonnegative CCA for Audiovisual Source Separation , 2007, 2007 IEEE Workshop on Machine Learning for Signal Processing.

[8]  N. H. Timm.  Applied Multivariate Analysis , 2002 .

[9]  Paul Horst,et al.  Relations amongm sets of measures , 1961 .

[10]  Mohamed Hanafi,et al.  Global optimality of the successive Maxbet algorithm , 2003 .

[11]  Gene H. Golub,et al.  Matrix computations , 1983 .

[12]  Moody T. Chu,et al.  On a Multivariate Eigenvalue Problem, Part I: Algebraic Theory and a Power Method , 1993, SIAM J. Sci. Comput..

[13]  L. Imhof.  Matrix Algebra and Its Applications to Statistics and Econometrics , 1998 .

[14]  R. Bhatia.  Positive Definite Matrices , 2007 .

[15]  Lei-Hong Zhang,et al.  ON A MULTIVARIATE EIGENVALUE PROBLEM: II. GLOBAL SOLUTIONS AND THE GAUSS-SEIDEL METHOD , 2008 .

[16]  Chong-sun Kim.  Canonical Analysis of Several Sets of Variables , 1973 .

[17]  Stephen J. Wright,et al.  Numerical Optimization (Springer Series in Operations Research and Financial Engineering) , 2000 .

[18]  Carl D. Meyer,et al.  Matrix Analysis and Applied Linear Algebra , 2000 .

[19]  H. Hotelling.  The most predictable criterion. , 1935 .

[20]  J. Berge,et al.  Generalized approaches to the maxbet problem and the maxdiff problem, with applications to canonical correlations , 1988 .

[21]  H. Hotelling.  Relations Between Two Sets of Variates , 1936 .