Designing structured tight frames via an alternating projection method

Tight frames, also known as general Welch-bound- equality sequences, generalize orthonormal systems. Numerous applications - including communications, coding, and sparse approximation- require finite-dimensional tight frames that possess additional structural properties. This paper proposes an alternating projection method that is versatile enough to solve a huge class of inverse eigenvalue problems (IEPs), which includes the frame design problem. To apply this method, one needs only to solve a matrix nearness problem that arises naturally from the design specifications. Therefore, it is the fast and easy to develop versions of the algorithm that target new design problems. Alternating projection will often succeed even if algebraic constructions are unavailable. To demonstrate that alternating projection is an effective tool for frame design, the paper studies some important structural properties in detail. First, it addresses the most basic design problem: constructing tight frames with prescribed vector norms. Then, it discusses equiangular tight frames, which are natural dictionaries for sparse approximation. Finally, it examines tight frames whose individual vectors have low peak-to-average-power ratio (PAR), which is a valuable property for code-division multiple-access (CDMA) applications. Numerical experiments show that the proposed algorithm succeeds in each of these three cases. The appendices investigate the convergence properties of the algorithm.

[1]  A. Cetin,et al.  Equiripple FIR filter design by the FFT algorithm , 1997, IEEE Signal Process. Mag..

[2]  Clifford Hildreth,et al.  A quadratic programming procedure , 1957 .

[3]  R. Tyrrell Rockafellar,et al.  Convex Analysis , 1970, Princeton Landmarks in Mathematics and Physics.

[4]  Allen Gersho,et al.  Vector quantization and signal compression , 1991, The Kluwer international series in engineering and computer science.

[5]  Inderjit S. Dhillon,et al.  Diametrical clustering for identifying anti-correlated gene clusters , 2003, Bioinform..

[6]  R. Dykstra.  An Algorithm for Restricted Least Squares Regression , 1983 .

[7]  Roy D. Yates,et al.  Iterative construction of optimum signature sequence sets in synchronous CDMA systems , 2001, IEEE Trans. Inf. Theory.

[8]  Suguru Arimoto,et al.  An algorithm for computing the capacity of arbitrary discrete memoryless channels , 1972, IEEE Trans. Inf. Theory.

[9]  Jinchao Xu,et al.  The method of alternating projections and the method of subspace corrections in Hilbert space , 2002 .

[10]  N. Higham.  Applications of matrix theory , edited by M. J. C. Gover and S. Barnett. Pp 324. £40. 1989. ISBN 0-19-853625-9 (Oxford University Press) , 1990, The Mathematical Gazette.

[11]  G. Golub,et al.  Structured inverse eigenvalue problems , 2002, Acta Numerica.

[12]  Frank Deutsch,et al.  The Method of Alternating Orthogonal Projections , 1992 .

[13]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[14]  W. Hogan.  Point-to-Set Maps in Mathematical Programming , 1973 .

[15]  Moody T. Chu,et al.  Constructing a Hermitian Matrix from Its Diagonal Entries and Eigenvalues , 1995, SIAM J. Matrix Anal. Appl..

[16]  Peter G. Casazza,et al.  Equal-Norm Tight Frames with Erasures , 2003, Adv. Comput. Math..

[17]  D. Donoho,et al.  Uncertainty principles and signal recovery , 1989 .

[18]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[19]  R. Plemmons,et al.  Structured low rank approximation , 2003 .

[20]  A. Calderbank,et al.  Z4‐Kerdock Codes, Orthogonal Spreads, and Extremal Euclidean Line‐Sets , 1997 .

[21]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[22]  D. Youla,et al.  Image Restoration by the Method of Convex Projections: Part 1ߞTheory , 1982, IEEE Transactions on Medical Imaging.

[23]  Mátyás A. Sustik,et al.  On the existence of equiangular tight frames , 2007 .

[24]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[25]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[26]  E. M. L. Beale,et al.  Nonlinear Programming: A Unified Approach. , 1970 .

[27]  R. Dykstra,et al.  A Method for Finding Projections onto the Intersection of Convex Sets in Hilbert Spaces , 1986 .

[28]  Y. Censor,et al.  Parallel Optimization:theory , 1997 .

[29]  O. Christensen.  An introduction to frames and Riesz bases , 2002 .

[30]  Venkat Anantharam,et al.  Optimal sequences and sum capacity of synchronous CDMA systems , 1999, IEEE Trans. Inf. Theory.

[31]  G. Stewart,et al.  A generalization of the Eckart-Young-Mirsky matrix approximation theorem , 1987 .

[32]  Akram Aldroubi,et al.  Nonuniform Sampling and Reconstruction in Shift-Invariant Spaces , 2001, SIAM Rev..

[33]  James L. Massey,et al.  Optimum sequence multisets for synchronous code-division multiple-access channels , 1994, IEEE Trans. Inf. Theory.

[34]  Paul Tseng,et al.  Dual coordinate ascent methods for non-strictly convex minimization , 1993, Math. Program..

[35]  W. Boothby.  An introduction to differentiable manifolds and Riemannian geometry , 1975 .

[36]  Robert W. Heath,et al.  Optimal CDMA signature sequences, inverse eigenvalue problems and alternating minimization , 2003, IEEE International Symposium on Information Theory, 2003. Proceedings..

[37]  N. Higham.  COMPUTING A NEAREST SYMMETRIC POSITIVE SEMIDEFINITE MATRIX , 1988 .

[38]  M. Sezan,et al.  Image Restoration by the Method of Convex Projections: Part 2-Applications and Numerical Results , 1982, IEEE Transactions on Medical Imaging.

[39]  Rémi Gribonval,et al.  Sparse representations in unions of bases , 2003, IEEE Trans. Inf. Theory.

[40]  E. Polak.  Method of Successive Projections for Finding a Common Point of Sets in Metric Spaces , 1990 .

[41]  N. Higham.  MATRIX NEARNESS PROBLEMS AND APPLICATIONS , 1989 .

[42]  Nicholas J. Higham,et al.  A Parallel Algorithm for Computing the Polar Decomposition , 1994, Parallel Comput..

[43]  T.J. Richardon,et al.  Multiple-antenna signal constellations for fading channels , 2000, 2000 IEEE International Symposium on Information Theory (Cat. No.00CH37060).

[44]  Rudolph E. Langer,et al.  On Numerical Approximation , 1959 .

[45]  N. Higham.  Computing the nearest correlation matrix—a problem from finance , 2002 .

[46]  Nachman Aronszajn.  Introduction to the theory of Hilbert spaces , 1950 .

[47]  Vivek K. Goyal,et al.  Quantized Overcomplete Expansions in IRN: Analysis, Synthesis, and Algorithms , 1998, IEEE Trans. Inf. Theory.

[48]  W. Schempp,et al.  Multivariate Approximation Theory IV , 1989 .

[49]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[50]  B. Green.  THE ORTHOGONAL APPROXIMATION OF AN OBLIQUE STRUCTURE IN FACTOR ANALYSIS , 1952 .

[51]  R. Gerchberg.  Super-resolution through Error Energy Reduction , 1974 .

[52]  Frank Deutsch.  The Alternating Method von Neumann , 1979 .

[53]  I. Olkin,et al.  Inequalities: Theory of Majorization and Its Applications , 1980 .

[54]  James A. Cadzow,et al.  Signal enhancement-a composite property mapping algorithm , 1988, IEEE Trans. Acoust. Speech Signal Process..

[55]  P. Schönemann,et al.  A generalized solution of the orthogonal procrustes problem , 1966 .

[56]  T. Strohmer.  Approximation of Dual Gabor Frames, Window Decay, and Wireless Communications , 2000, math/0010244.

[57]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[58]  P. Huard,et al.  Point-to-set maps and mathematical programming , 1979 .

[59]  Christopher Rose.  CDMA Codeword optimization: Interference avoidance and convergence via class warfare , 2001, IEEE Trans. Inf. Theory.

[60]  M. Chu.  Numerical Methods for Inverse Singular Value Problems , 1992 .

[61]  Heinz H. Bauschke,et al.  Accelerating the convergence of the method of alternating projections , 2003 .

[62]  Il-Min Kim,et al.  Existence and construction of noncoherent unitary space-time codes , 2002, IEEE Trans. Inf. Theory.

[63]  J. Tropp,et al.  CDMA signature sequences with low peak-to-average-power ratio via alternating projection , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[64]  E. Straus,et al.  On the approximation of a function of several variables by the sum of functions of fewer variables. , 1951 .

[65]  Robert W. Heath,et al.  On quasi-orthogonal signatures for CDMA systems , 2006, IEEE Transactions on Information Theory.

[66]  Dilip V. Sarwate.  Meeting the Welch Bound with Equality , 1998, SETA.

[67]  W. L. Miranker,et al.  The recovery of distorted band-limited signals , 1961 .

[68]  Vivek K Goyal.  Quantized Overcomplete Expansions : Analysis , Synthesis and Algorithms , 1995 .

[69]  Will Light,et al.  Approximation Theory in Tensor Product Spaces , 1985 .

[70]  Joel A. Tropp,et al.  Topics in sparse approximation , 2004 .

[71]  Lizhong Zheng,et al.  Communication on the Grassmann manifold: A geometric approach to the noncoherent multiple-antenna channel , 2002, IEEE Trans. Inf. Theory.

[72]  David W. Lewis,et al.  Matrix theory , 1991 .

[73]  Howard L. Weinert,et al.  Error bounds for the method of alternating projections , 1988, Math. Control. Signals Syst..

[74]  Jan de Leeuw,et al.  Block-relaxation Algorithms in Statistics , 1994 .

[75]  Robert R. Meyer,et al.  Sufficient Conditions for the Convergence of Monotonic Mathematical Programming Algorithms , 1976, J. Comput. Syst. Sci..

[76]  Kennan T. Smith,et al.  Practical and mathematical aspects of the problem of reconstructing objects from radiographs , 1977 .

[77]  Inderjit S. Dhillon,et al.  Concept Decompositions for Large Sparse Text Data Using Clustering , 2004, Machine Learning.

[78]  Martin Vetterli,et al.  Oversampled filter banks , 1998, IEEE Trans. Signal Process..

[79]  John J. Benedetto,et al.  Finite Normalized Tight Frames , 2003, Adv. Comput. Math..

[80]  Thomas Strohmer,et al.  GRASSMANNIAN FRAMES WITH APPLICATIONS TO CODING AND COMMUNICATION , 2003, math/0301135.

[81]  Lloyd R. Welch,et al.  Lower bounds on the maximum cross correlation of signals (Corresp.) , 1974, IEEE Trans. Inf. Theory.

[82]  Sennur Ulukus,et al.  Iterative Joint Optimization of CDMA Signature Sequences and Receiver Filters , 2002 .

[83]  Norbert Wiener,et al.  On the factorization of matrices , 1955 .

[84]  Hans-Hermann Bock,et al.  Information Systems and Data Analysis , 1994 .

[85]  K. Gröchenig,et al.  NON-UNIFORM SAMPLING IN SHIFT-INVARIANT SPACES , 2007 .

[86]  Kenneth R. Driessel,et al.  The projected gradient methods for least squares matrix approximations with spectral constraints , 1990 .

[87]  N. Higham.  Computing the polar decomposition with applications , 1986 .

[88]  N. J. A. Sloane,et al.  Packing Lines, Planes, etc.: Packings in Grassmannian Spaces , 1996, Exp. Math..

[89]  Shih-Ping Han,et al.  A successive projection method , 1988, Math. Program..

[90]  A. Ostrowski.  Solution of equations and systems of equations , 1967 .

[91]  Roy D. Yates,et al.  Wireless systems and interference avoidance , 2002, IEEE Trans. Wirel. Commun..

[92]  Venkat Anantharam,et al.  Ensuring convergence of the MMSE iteration for interference avoidance to the global optimum , 2003, IEEE Trans. Inf. Theory.

[93]  Christopher Heil,et al.  Integral Operators, Pseudodifferential Operators, and Gabor Frames , 2003 .

[94]  William Kile Glunt,et al.  The nearest 'doubly stochastic' matrix to a real matrix with the same first moment , 1998, Numer. Linear Algebra Appl..

[95]  Vincent K. N. Lau.  On the analysis of peak-to-average ratio (PAR) for IS95 and CDMA2000 systems , 2000, IEEE Trans. Veh. Technol..

[96]  N.J.A. Sloane,et al.  Packing Planes in Four Dimensions and Other Mysteries , 2002, math/0208017.

[97]  Thomas L. Marzetta,et al.  Systematic design of unitary space-time constellations , 2000, IEEE Trans. Inf. Theory.

[98]  M. Chu.  Numerical methods for inverse singular value problems3 , 1992 .

[99]  G. Golub,et al.  Acta Numerica 2002: Structured inverse eigenvalue problems , 2002 .

[100]  Z. Opial.  Weak convergence of the sequence of successive approximations for nonexpansive mappings , 1967 .

[101]  W. Cheney,et al.  Proximity maps for convex sets , 1959 .

[102]  Y. Censor,et al.  Parallel Optimization: Theory, Algorithms, and Applications , 1997 .

[103]  Robert W. Heath,et al.  Finite-step algorithms for constructing optimal CDMA signature sequences , 2004, IEEE Transactions on Information Theory.

[104]  Gene H. Golub,et al.  Matrix computations , 1983 .

[105]  J. C. Fiorot,et al.  Composition and union of general algorithms of optimization , 1979 .

[106]  G. Dantzig,et al.  On the continuity of the minimum set of a continuous function , 1967 .

[107]  Vivek K. Goyal,et al.  Generalized multiple description coding with correlating transforms , 2001, IEEE Trans. Inf. Theory.

[108]  Moody T. Chu,et al.  Inverse Eigenvalue Problems , 1998, SIAM Rev..

[109]  Heinz H. Bauschke,et al.  On Projection Algorithms for Solving Convex Feasibility Problems , 1996, SIAM Rev..

[110]  Richard E. Blahut,et al.  Computation of channel capacity and rate-distortion functions , 1972, IEEE Trans. Inf. Theory.

[111]  L. Bregman.  The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming , 1967 .

[112]  Axel Ruhe.  Closest normal matrix finally found! , 1987 .

[113]  L. Collatz,et al.  Approximation by functions of fewer variables , 1972 .

[114]  G. Alexits.  Approximation theory , 1983 .

[115]  Venkat Anantharam,et al.  Optimal sequences for CDMA under colored noise: A Schur-saddle function property , 2002, IEEE Trans. Inf. Theory.