Optimization algorithms exploiting unitary constraints

This paper presents novel algorithms that iteratively converge to a local minimum of a real-valued function f (X) subject to the constraint that the columns of the complex-valued matrix X are mutually orthogonal and have unit norm. The algorithms are derived by reformulating the constrained optimization problem as an unconstrained one on a suitable manifold. This significantly reduces the dimensionality of the optimization problem. Pertinent features of the proposed framework are illustrated by using the framework to derive an algorithm for computing the eigenvector associated with either the largest or the smallest eigenvalue of a Hermitian matrix.

[1]  W. Boothby.  An introduction to differentiable manifolds and Riemannian geometry , 1975 .

[2]  R. Brockett,et al.  Dynamical systems that sort lists, diagonalize matrices and solve linear programming problems , 1988, Proceedings of the 27th IEEE Conference on Decision and Control.

[3]  Donald W. Tufts,et al.  Two algorithms for fast approximate subspace tracking , 1999, IEEE Trans. Signal Process..

[4]  A. V. D. Veen.  Algebraic methods for deterministic blind beamforming , 1998, Proc. IEEE.

[5]  Lang Tong,et al.  Indeterminacy and identifiability of blind identification , 1991 .

[6]  Robert E. Mahony,et al.  The Geometry of the Newton Method on Non-Compact Lie Groups , 2002, J. Glob. Optim..

[7]  Peter Strobach.  Square-root QR inverse iteration for tracking the minor subspace , 2000, IEEE Trans. Signal Process..

[8]  Vwani P. Roychowdhury,et al.  An adaptive quasi-Newton algorithm for eigensubspace estimation , 2000, IEEE Trans. Signal Process..

[9]  Yingbo Hua,et al.  Convolutive reduced rank Wiener filtering , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[10]  Eric Moulines,et al.  A blind source separation technique using second-order statistics , 1997, IEEE Trans. Signal Process..

[11]  R. Mahony.  Optimization algorithms on homogeneous spaces: with application in linear systems theory , 1995, Journal and proceedings of the Royal Society of New South Wales.

[12]  Robert E. Mahony,et al.  The geometry of weighted low-rank approximations , 2003, IEEE Trans. Signal Process..

[13]  Tapan K. Sarkar,et al.  A survey of conjugate gradient algorithms for solution of extreme eigen-problems of a symmetric matrix , 1989, IEEE Trans. Acoust. Speech Signal Process..

[14]  G. Golub,et al.  Tracking a few extreme singular values and vectors in signal processing , 1990, Proc. IEEE.

[15]  J. Cardoso,et al.  Blind beamforming for non-gaussian signals , 1993 .

[16]  U. Helmke,et al.  Optimization and Dynamical Systems , 1994, Proceedings of the IEEE.

[17]  Elijah Polak,et al.  Optimization: Algorithms and Consistent Approximations , 1997 .

[18]  Alan Edelman,et al.  The Geometry of Algorithms with Orthogonality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[19]  Sun-Yuan Kung,et al.  On gradient adaptation with unit-norm constraints , 2000, IEEE Trans. Signal Process..

[20]  T. Kailath,et al.  Least squares type algorithm for adaptive implementation of Pisarenko's harmonic retrieval method , 1982 .

[21]  C. Udriste,et al.  Convex Functions and Optimization Methods on Riemannian Manifolds , 1994 .

[22]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[23]  R. Mahony.  The constrained newton method on a Lie group and the symmetric eigenvalue problem , 1996 .

[24]  Soura Dasgupta,et al.  Adaptive estimation of eigensubspace , 1995, IEEE Trans. Signal Process..

[25]  M. Wax,et al.  A least-squares approach to joint diagonalization , 1997, IEEE Signal Processing Letters.

[26]  Jar-Ferr Yang,et al.  Adaptive eigensubspace algorithms for direction or frequency estimation and tracking , 1988, IEEE Trans. Acoust. Speech Signal Process..

[27]  Arogyaswami Paulraj,et al.  An analytical constant modulus algorithm , 1996, IEEE Trans. Signal Process..

[28]  James P. Reilly,et al.  Blind source separation of convolved sources by joint approximate diagonalization of cross-spectral density matrices , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[29]  John B. Moore,et al.  Gradient algorithms for principal component analysis , 1996, The Journal of the Australian Mathematical Society. Series B. Applied Mathematics.

[30]  Anthony J. Weiss,et al.  Estimating frequencies of exponentials in noise using joint diagonalization , 1999, IEEE Trans. Signal Process..

[31]  R. Brockett.  Dynamical systems that sort lists, diagonalize matrices, and solve linear programming problems , 1991 .

[32]  J. H. Wilkinson.  The algebraic eigenvalue problem , 1966 .

[33]  Benoît Champagne,et al.  Plane rotation-based EVD updating schemes for efficient subspace tracking , 1998, IEEE Trans. Signal Process..

[34]  Daniel J. Rabideau,et al.  Fast, rank adaptive subspace tracking and applications , 1996, IEEE Trans. Signal Process..

[35]  Paul S. Wang,et al.  Weighted Low-Rank Approximation of General Complex Matrices and Its Application in the Design of 2-D Digital Filters , 1997 .

[36]  I. Holopainen.  Riemannian Geometry , 1927, Nature.

[37]  Robert Schreiber,et al.  Implementation of adaptive array algorithms , 1986, IEEE Trans. Acoust. Speech Signal Process..