Applications of Large Random Matrices in Communications Engineering

This work gives an overview of analytic tools for the design, analysis, and modelling of communication systems which can be described by linear vector channels such as y = Hx+z where the number of components in each vector is large. Tools from probability theory, operator algebra, and statistical physics are reviewed. The survey of analytical tools is complemented by examples of applications in communications engineering. Asymptotic eigenvalue distributions of many classes of random matrices are given. The treatment includes the problem of moments and the introduction of the Stieltjes transform. Free probability theory, which evolved from non-commutative operator algebras, is explained from a probabilistic point of view in order to better fit the engineering community. For that purpose freeness is defined without reference to non-commutative algebras. The treatment includes additive and multiplicative free convolution, the R-transform, the S-transform, and the free central limit theorem. The replica method developed in statistical physics for the purpose of analyzing spin glasses is reviewed from the viewpoint of its applications in communications engineering. Correspondences between free energy and mutual information as well as energy functions and detector metrics are established. These analytic tools are applied to the design and the analysis of linear multiuser detectors, the modelling of scattering in communication channels with dual antennas arrays, and the analysis of optimal detection for communication via code-division multiple-access and/or dual antenna array channels.

[1]  Robert F. H. Fischer,et al.  Precoding and Signal Shaping for Digital Transmission , 2002 .

[2]  F. Riesz,et al.  Sur Les Valeurs Moyennes Des Fonctions , 1930 .

[3]  F. Hiai,et al.  The semicircle law, free random variables, and entropy , 2006 .

[4]  J. W. Silverstein,et al.  On the empirical distribution of eigenvalues of a class of large dimensional random matrices , 1995 .

[5]  John M. Cioffi,et al.  Gigabit DSL , 2007, IEEE Transactions on Communications.

[6]  Philippe Loubaton,et al.  Asymptotic analysis of reduced rank Wiener filters , 2003, Proceedings 2003 IEEE Information Theory Workshop (Cat. No.03EX674).

[7]  K. Adkins Theory of spin glasses , 1974 .

[8]  Sergio Verdú,et al.  Replica analysis of large-system CDMA , 2003, Proceedings 2003 IEEE Information Theory Workshop (Cat. No.03EX674).

[9]  D. Thouless,et al.  Stability of the Sherrington-Kirkpatrick solution of a spin glass model , 1978 .

[10]  Ralf R. Müller,et al.  Vector Precoding for Gaussian MIMO Broadcast Channels: Impact of Replica Symmetry Breaking , 2010, IEEE Transactions on Information Theory.

[11]  Stephan ten Brink,et al.  A close-to-capacity dirty paper coding scheme , 2004, IEEE Transactions on Information Theory.

[12]  Koujin Takeda,et al.  Analysis of CDMA systems that are characterized by eigenvalue spectrum , 2006, ArXiv.

[13]  Bruno O. Shubert,et al.  Random variables and stochastic processes , 1979 .

[14]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[15]  Ralf R. Müller,et al.  Asynchronous CDMA Systems With Random Spreading—Part II: Design Criteria , 2009, IEEE Transactions on Information Theory.

[16]  D. Stein Introduction to the Replica Theory of Disordered Statistical Systems , 2002 .

[17]  R. Muller,et al.  Minimum probability of error and channel capacity in large dual antenna array systems , 2003, 10th International Conference on Telecommunications, 2003. ICT 2003..

[18]  Sergio Verdú,et al.  Randomly spread CDMA: asymptotics via statistical physics , 2005, IEEE Transactions on Information Theory.

[19]  Nicolas Sourlas,et al.  Spin-glass models as error-correcting codes , 1989, Nature.

[20]  Bernard Fino,et al.  Multiuser detection: , 1999, Ann. des Télécommunications.

[21]  Mark Kac On Certain Toeplitz-Like Matrices and Their Relation to the Problem of Lattice Vibrations , 2013 .

[22]  Y. Yin Limiting spectral distribution for a class of random matrices , 1986 .

[23]  Ralf R. Müller,et al.  On the asymptotic eigenvalue distribution of concatenated vector-valued fading channels , 2002, IEEE Trans. Inf. Theory.

[24]  Elias S. Manolakos,et al.  Hopfield neural network implementation of the optimal CDMA multiuser detector , 1996, IEEE Trans. Neural Networks.

[25]  M. Talagrand The parisi formula , 2006 .

[26]  R. R. Muller,et al.  Performance analysis of large dual antenna array systems with binary modulation , 2003, IEEE International Symposium on Information Theory, 2003. Proceedings..

[27]  Max H. M. Costa,et al.  Writing on dirty paper , 1983, IEEE Trans. Inf. Theory.

[28]  J. Romberg,et al.  Imaging via Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[29]  Vi︠a︡cheslav Leonidovich Girko,et al.  Theory of random determinants , 1990 .

[30]  Sergio Verdú,et al.  Optimum asymptotic multiuser efficiency of randomly spread CDMA , 2000, IEEE Trans. Inf. Theory.

[31]  Nicolas Sourlas Statistical mechanics and capacity-approaching error-correcting codes , 2001 .

[32]  Roland Speicher,et al.  On slow-fading non-separable correlation MIMO systems , 2007, ArXiv.

[33]  Thomas L. Marzetta,et al.  Multiple-antennas and isotropically random unitary inputs: The received signal density in closed form , 2002, IEEE Trans. Inf. Theory.

[34]  Ralf R. Müller,et al.  Design and analysis of low-complexity interference mitigation on vector channels , 2001, IEEE J. Sel. Areas Commun..

[35]  John G. Proakis,et al.  Probability, random variables and stochastic processes , 1985, IEEE Trans. Acoust. Speech Signal Process..

[36]  Shlomo Shamai,et al.  Spectral Efficiency of CDMA with Random Spreading , 1999, IEEE Trans. Inf. Theory.

[37]  Ralf R. Müller,et al.  CDMA Systems With Correlated Spatial Diversity: A Generalized Resource Pooling Result , 2007, IEEE Transactions on Information Theory.

[38]  J.E. Mazo,et al.  Digital communications , 1985, Proceedings of the IEEE.

[39]  David Tse Multiuser Receivers, Random Matrices and Free Probability , 1999 .

[40]  T. Stieltjes Recherches sur les fractions continues , 1995 .

[41]  西森 秀稔 Statistical physics of spin glasses and information processing : an introduction , 2001 .

[42]  Philippe Loubaton,et al.  MMSE analysis of certain large isometric random precoded systems , 2003, IEEE Trans. Inf. Theory.

[43]  J. Andersen,et al.  Power Distributions Revisited , 2002 .

[44]  Shlomo Shamai,et al.  On Certain Large Random Hermitian Jacobi Matrices With Applications to Wireless Communications , 2009, IEEE Transactions on Information Theory.

[45]  Shlomo Shamai,et al.  On Information Rates of the Fading Wyner Cellular Model via the Thouless Formula for the Strip , 2010, IEEE Transactions on Information Theory.

[46]  S. Verdú,et al.  Asymptotic eigenvalue moments for linear multiuser detection , 2001, Conference Record of Thirty-Fifth Asilomar Conference on Signals, Systems and Computers (Cat.No.01CH37256).

[47]  R. Remmert,et al.  Theory of Complex Functions , 1990 .

[48]  S. Thorbjørnsen,et al.  Mixed Moments of Voiculescu's Gaussian Random Matrices , 2000 .

[49]  Ralf R. Müller,et al.  Vector Precoding for Wireless MIMO Systems: A Replica Analysis , 2007, ArXiv.

[50]  Iain B. Collings,et al.  Large system analysis of linear multistage parallel interference cancellation , 2002, IEEE Trans. Commun..

[51]  Alexandru Nica,et al.  Free random variables , 1992 .

[52]  Ralf R. Müller,et al.  Channel capacity and minimum probability of error in large dual antenna array systems with binary modulation , 2003, IEEE Trans. Signal Process..

[53]  G. Parisi A sequence of approximated solutions to the S-K model for spin glasses , 1980 .

[54]  H. Miyakawa,et al.  Matched-Transmission Technique for Channels With Intersymbol Interference , 1972, IEEE Trans. Commun..

[55]  Dongning Guo,et al.  Spectral efficiency of large-system CDMA via statistical physics , 2003 .

[56]  Ralf R. Müller,et al.  A systematic approach to multistage detectors in multipath fading channels , 2005, IEEE Transactions on Information Theory.

[57]  D. Voiculescu Limit laws for Random matrices and free products , 1991 .

[58]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[59]  Holger Rauhut,et al.  Compressive Estimation of Doubly Selective Channels in Multicarrier Systems: Leakage Effects and Sparsity-Enhancing Processing , 2009, IEEE Journal of Selected Topics in Signal Processing.

[60]  Ralf R. Müller,et al.  Channel modelling of MU-MIMO systems by quaternionic free probability , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[61]  J. W. Silverstein,et al.  ON THE SIGNAL-TO-INTERFERENCE RATIO OF CDMA SYSTEMS IN WIRELESS COMMUNICATIONS , 2007, math/0702888.

[62]  S. Kirkpatrick,et al.  Solvable Model of a Spin-Glass , 1975 .

[63]  Wolfgang H. Gerstacker,et al.  On the capacity loss due to separation of detection and decoding in large CDMA systems , 2002, Proceedings of the IEEE Information Theory Workshop.

[64]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[65]  Ralf R. Müller,et al.  On the capacity loss due to separation of detection and decoding , 2004, IEEE Transactions on Information Theory.

[66]  Antonia Maria Tulino,et al.  Performance of space-time codes for a large number of antennas , 2002, IEEE Trans. Inf. Theory.

[67]  Olivier Lévêque,et al.  Diversity-Multiplexing Tradeoff for the MIMO Static Half-Duplex Relay , 2010, IEEE Transactions on Information Theory.

[68]  Ralf R. M̈uller On Channel Capacity , Uncoded Error Probability , ML-Detection and Spin Glasses , 2007 .

[69]  Michael L. Honig,et al.  Performance of reduced-rank linear interference suppression , 2001, IEEE Trans. Inf. Theory.

[70]  Ralf R. Müller,et al.  Vector Precoding for Wireless MIMO Systems and its Replica Analysis , 2007, IEEE Journal on Selected Areas in Communications.

[71]  Simon R. Saunders,et al.  Antennas and Propagation for Wireless Communication Systems , 1999 .

[72]  Mérouane Debbah,et al.  Asymptotic Behaviour of Random Vandermonde Matrices with Entries on the Unit Circle , 2008, 0802.3570.

[73]  A. Jarosz Summing free unitary random matrices. , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[74]  Ralf R. Müller,et al.  THE REPLICA METHOD IN MULTIUSER COMMUNICATIONS , 2009 .

[75]  Alain Glavieux,et al.  Reflections on the Prize Paper : "Near optimum error-correcting coding and decoding: turbo codes" , 1998 .

[76]  J. W. Silverstein Strong convergence of the empirical distribution of eigenvalues of large dimensional random matrices , 1995 .

[77]  A. Lee Swindlehurst,et al.  A vector-perturbation technique for near-capacity multiantenna multiuser communication-part II: perturbation , 2005, IEEE Transactions on Communications.

[78]  Giuseppe Caire,et al.  The optimal received power distribution for IC-based iterative multiuser joint decoders , 2001 .

[79]  Philippe Loubaton,et al.  Asymptotic Analysis of Reduced-Rank Chip-Level MMSE Equalizers in the Downlink of CDMA Systems , 2004, IEEE Transactions on Signal Processing.

[80]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[81]  Maciej A. Nowak,et al.  Random Hermitian versus random non-Hermitian operators—unexpected links , 2006 .

[82]  A. Guionnet,et al.  A Fourier view on the R-transform and related asymptotics of spherical integrals , 2005 .

[83]  B. Lautrup,et al.  Products of random matrices. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[84]  Ting Sun,et al.  Single-pixel imaging via compressive sampling , 2008, IEEE Signal Process. Mag..

[85]  Shlomo Shamai,et al.  Support Recovery With Sparsely Sampled Free Random Matrices , 2011, IEEE Transactions on Information Theory.

[86]  J.-M. Chaufray,et al.  Asymptotic analysis of optimum and sub-optimum CDMA downlink MMSE receivers , 2002, Seventh International Symposium on Signal Processing and Its Applications, 2003. Proceedings..

[87]  Harish-Chandra Fourier Transforms on a Semisimple Lie Algebra II , 1957 .

[88]  Massimo Fornasier,et al.  Compressive Sensing and Structured Random Matrices , 2010 .

[89]  David Tse,et al.  Resource pooling and effective bandwidths in CDMA networks with multiuser receivers and spatial diversity , 2001, IEEE Trans. Inf. Theory.

[90]  M. Tomlinson New automatic equaliser employing modulo arithmetic , 1971 .

[91]  L. Litwin,et al.  Error control coding , 2001 .

[92]  Claude E. Shannon,et al.  A Mathematical Theory of Communications , 1948 .

[93]  Shlomo Shamai,et al.  Nested linear/Lattice codes for structured multiterminal binning , 2002, IEEE Trans. Inf. Theory.

[94]  Shlomo Shamai,et al.  The impact of frequency-flat fading on the spectral efficiency of CDMA , 2001, IEEE Trans. Inf. Theory.

[95]  Michael L. Honig,et al.  Advances in Multiuser Detection , 2009 .

[96]  Aaron D. Wyner,et al.  Shannon-theoretic approach to a Gaussian cellular multiple-access channel , 1994, IEEE Trans. Inf. Theory.

[97]  Shlomo Shamai,et al.  Capacity of Channels With Frequency-Selective and Time-Selective Fading , 2010, IEEE Transactions on Information Theory.

[98]  James L. Massey,et al.  Proper complex random processes with applications to information theory , 1993, IEEE Trans. Inf. Theory.

[99]  V. Marčenko,et al.  DISTRIBUTION OF EIGENVALUES FOR SOME SETS OF RANDOM MATRICES , 1967 .

[100]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[101]  Sundeep Rangan,et al.  Asymptotic Analysis of MAP Estimation via the Replica Method and Applications to Compressed Sensing , 2009, IEEE Transactions on Information Theory.

[102]  Holger Dette,et al.  Random block matrices generalizing the classical Jacobi and Laguerre ensembles , 2010, J. Multivar. Anal..

[103]  S. Kak Information, physics, and computation , 1996 .

[104]  S. Kirkpatrick,et al.  Infinite-ranged models of spin-glasses , 1978 .

[105]  S. Thorbjjrnsen Mixed Moments of Voiculescu's Gaussian Random Matrices , 1999 .

[106]  P. Deift Orthogonal Polynomials and Random Matrices: A Riemann-Hilbert Approach , 2000 .

[107]  G. Caire,et al.  Efficient implementation of iterative multiuser decoding , 2002, Proceedings IEEE International Symposium on Information Theory,.

[108]  Nicolas Macris,et al.  Sharp Bounds for Optimal Decoding of Low-Density Parity-Check Codes , 2008, IEEE Transactions on Information Theory.

[109]  Rüdiger L. Urbanke,et al.  Modern Coding Theory , 2008 .

[110]  D. Thouless,et al.  Spherical Model of a Spin-Glass , 1976 .

[111]  U. Haagerup,et al.  Brown's Spectral Distribution Measure for R-Diagonal Elements in Finite von Neumann Algebras☆ , 2000 .

[112]  Jean-Jacques Fuchs,et al.  On sparse representations in arbitrary redundant bases , 2004, IEEE Transactions on Information Theory.

[113]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[114]  Philippe Loubaton,et al.  Asymptotic analysis of optimum and suboptimum CDMA downlink MMSE receivers , 2004, IEEE Trans. Inf. Theory.

[115]  Donald L. Schilling,et al.  Multistage linear receivers for DS-CDMA systems , 1996, Int. J. Wirel. Inf. Networks.

[116]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[117]  Shlomo Shamai,et al.  The Capacity Region of the Gaussian Multiple-Input Multiple-Output Broadcast Channel , 2006, IEEE Transactions on Information Theory.

[118]  C. Itzykson,et al.  The planar approximation. II , 1980 .

[119]  Alexander Vardy,et al.  Closest point search in lattices , 2002, IEEE Trans. Inf. Theory.

[120]  Toshiyuki Tanaka,et al.  A statistical-mechanics approach to large-system analysis of CDMA multiuser detectors , 2002, IEEE Trans. Inf. Theory.

[121]  P. Hilton,et al.  Catalan Numbers, Their Generalization, and Their Uses , 1991 .

[122]  Andrea Montanari,et al.  Tight bounds for LDPC and LDGM codes under MAP decoding , 2004, IEEE Transactions on Information Theory.

[123]  Jun Yin,et al.  Delocalization and Diffusion Profile for Random Band Matrices , 2012, 1205.5669.

[124]  Christodoulou Antennas and Propagation for Wireless Communication , 2006 .

[125]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[126]  David Tse,et al.  Linear Multiuser Receivers: Effective Interference, Effective Bandwidth and User Capacity , 1999, IEEE Trans. Inf. Theory.

[127]  O. Axelsson Iterative solution methods , 1995 .

[128]  Z. Burda,et al.  Multiplication law and S transform for non-Hermitian random matrices. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[129]  Candice King,et al.  Fundamentals of wireless communications , 2013, 2013 IEEE Rural Electric Power Conference (REPC).

[130]  D. Saad,et al.  Statistical mechanics of low-density parity-check codes , 2004 .

[131]  S. Verdú,et al.  Multiuser Detection and Statistical Mechanics , 2003 .

[132]  H. Nishimori Statistical Physics of Spin Glasses and Information Processing , 2001 .

[133]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[134]  Giuseppe Caire,et al.  Density evolution and power profile optimization for iterative multiuser decoders based on individually optimum multiuser detectors , 2002 .

[135]  Alexander Vardy,et al.  The turbo decoding algorithm and its phase trajectories , 2001, IEEE Trans. Inf. Theory.

[136]  Thomas Strohmer,et al.  High-Resolution Radar via Compressed Sensing , 2008, IEEE Transactions on Signal Processing.

[137]  Jamie S. Evans,et al.  Large system performance of linear multiuser receivers in multipath fading channels , 2000, IEEE Trans. Inf. Theory.

[138]  Ralf R. Müller,et al.  A random matrix model of communication via antenna arrays , 2002, IEEE Trans. Inf. Theory.

[139]  J. Scott Goldstein,et al.  Reduced-rank adaptive filtering , 1997, IEEE Trans. Signal Process..

[140]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[141]  Louis L. Scharf,et al.  A Multistage Representation of the Wiener Filter Based on Orthogonal Projections , 1998, IEEE Trans. Inf. Theory.

[142]  Iain B. Collings,et al.  Optimal multistage linear multiuser receivers , 2005, IEEE Transactions on Wireless Communications.

[143]  R. Muller,et al.  Confirmation of random matrix model for the antenna array channel by indoor measurements , 2001, IEEE Antennas and Propagation Society International Symposium. 2001 Digest. Held in conjunction with: USNC/URSI National Radio Science Meeting (Cat. No.01CH37229).

[144]  M. Bona Partitions with k Crossings , 1999 .

[145]  Geert Leus,et al.  OFDM Communications over Time-Varying Channels , 2011 .

[146]  S. Moshavi,et al.  Multi-user detection for DS-CDMA communications , 1996, IEEE Commun. Mag..