Applications of the Lindeberg Principle in Communications and Statistical Learning

We use a generalization of the Lindeberg principle developed by S. Chatterjee to prove universality properties for various problems in communications, statistical learning and random matrix theory. We also show that these systems can be viewed as the limiting case of a properly defined sparse system. The latter result is useful when the sparse systems are easier to analyze than their dense counterparts. The list of problems we consider is by no means exhaustive. We believe that the ideas can be used in many other problems relevant for information theory.

[1]  J. W. Silverstein,et al.  Spectral Analysis of Large Dimensional Random Matrices , 2009 .

[2]  Harry Kesten,et al.  Symmetric random walks on groups , 1959 .

[3]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[4]  Toshiyuki Tanaka,et al.  A statistical-mechanics approach to large-system analysis of CDMA multiuser detectors , 2002, IEEE Trans. Inf. Theory.

[5]  J. Cardy.  Scaling and Renormalization in Statistical Physics , 1996 .

[6]  S. Chatterjee.  A generalization of the Lindeberg principle , 2005, math/0508519.

[7]  A. Grant,et al.  Randomly selected spreading sequences for coded CDMA , 1996, Proceedings of ISSSTA'95 International Symposium on Spread Spectrum Techniques and Applications.

[8]  C. Tracy,et al.  Introduction to Random Matrices , 1992, hep-th/9210073.

[9]  A.M. Sayeed,et al.  Maximizing MIMO Capacity in Sparse Multipath With Reconfigurable Antenna Arrays , 2007, IEEE Journal of Selected Topics in Signal Processing.

[10]  Yoav Seginer,et al.  The Expected Norm of Random Matrices , 2000, Combinatorics, Probability and Computing.

[11]  F. Guerra,et al.  The High Temperature Region of the Viana–Bray Diluted Spin Glass Model , 2003, cond-mat/0302401.

[12]  Angel Lozano,et al.  Capacity-achieving Input Covariance for Correlated Multi-Antenna Channels , 2003 .

[13]  M. Talagrand.  Spin glasses : a challenge for mathematicians : cavity and mean field models , 2003 .

[14]  B. McKay.  The expected eigenvalue distribution of a large regular graph , 1981 .

[15]  David L. Donoho,et al.  Observed universality of phase transitions in high-dimensional geometry, with implications for modern data analysis and signal processing , 2009, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[16]  Emre Telatar,et al.  Capacity of Multi-antenna Gaussian Channels , 1999, Eur. Trans. Telecommun..

[17]  Chuang Liu,et al.  Scaling and Renormalization , 2002 .

[18]  D. Donoho,et al.  Counting faces of randomly-projected polytopes when the projection radically lowers dimension , 2006, math/0607364.

[19]  S. Chatterjee.  A simple invariance theorem , 2005, math/0508213.

[20]  Sergio Verdu,et al.  Multiuser Detection , 1998 .

[21]  S. Kak.  Information, physics, and computation , 1996 .

[22]  P. Carmona,et al.  Universality in Sherrington–Kirkpatrick's spin glass model , 2004, math/0403359.

[23]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[24]  V. Marčenko,et al.  DISTRIBUTION OF EIGENVALUES FOR SOME SETS OF RANDOM MATRICES , 1967 .

[25]  Michel Talagrand,et al.  Gaussian averages, Bernoulli averages, and Gibbs' measures , 2002, Random Struct. Algorithms.

[26]  Nicolas Macris,et al.  Tight Bounds on the Capacity of Binary Input Random CDMA Systems , 2008, IEEE Transactions on Information Theory.

[27]  Andrea Montanari,et al.  Analysis of Belief Propagation for Non-Linear Problems: The Example of CDMA (or: How to Prove Tanaka's Formula) , 2006, 2006 IEEE Information Theory Workshop - ITW '06 Punta del Este.