Threshold Effects in Parameter Estimation as Phase Transitions in Statistical Mechanics

Threshold effects in the estimation of parameters of nonlinearly modulated, continuous-time, wide-band waveforms, are examined from a statistical physics perspective. These threshold effects are shown to be analogous to phase transitions of certain disordered physical systems in thermal equilibrium. The main message, in this work, is in demonstrating that this physical point of view may be insightful for understanding the interactions between two or more parameters to be estimated, from the aspects of the threshold effect.

[1]  Alexander Barg,et al.  Random codes: Minimum distances and error exponents , 2002, IEEE Trans. Inf. Theory.

[2]  I. M. Jacobs,et al.  Principles of Communication Engineering , 1965 .

[3]  N. Sourlas.  Spin Glasses, Error-Correcting Codes and Finite-Temperature Decoding , 1994 .

[4]  Fredrik Hekland.  On the Design and Analysis of Shannon-Kotel'nikov Mappings for Joint-Source-Channel Coding , 2007 .

[5]  L. Shepp.  Radon-Nikodym Derivatives of Gaussian Measures , 1966 .

[6]  Rodger E. Ziemer,et al.  Principles of Communication , 2001 .

[7]  L. Shepp,et al.  Sample behavior of Gaussian processes , 1972 .

[8]  Ruján.  Finite temperature error-correcting codes. , 1993, Physical review letters.

[9]  Y. Iba.  The Nishimori line and Bayesian statistics , 1998, cond-mat/9809190.

[10]  D. Slepian.  The one-sided barrier problem for Gaussian noise , 1962 .

[11]  M. Mézard,et al.  Information, Physics, and Computation , 2009 .

[12]  Shlomo Shamai,et al.  Statistical Physics of Signal Estimation in Gaussian Noise: Theory and Examples of Phase Transitions , 2008, IEEE Transactions on Information Theory.

[13]  Pål Anders Floor.  On the Theory of Shannon-Kotel'nikov Mappings in Joint Source-Channel Coding , 2008 .

[14]  A. Montanari.  The glassy phase of Gallager codes , 2001, cond-mat/0104079.

[15]  B. Derrida.  Random-Energy Model: Limit of a Family of Disordered Models , 1980 .

[16]  Ian F. Blake,et al.  Level-crossing problems for random processes , 1973, IEEE Trans. Inf. Theory.

[17]  S. Franz,et al.  Dynamic phase transition for decoding algorithms. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[18]  Carl W. Helstrom,et al.  A bound on the probability that a Gaussian process exceeds a given function (Corresp.) , 1969, IEEE Trans. Inf. Theory.

[19]  Neri Merhav,et al.  Relations Between Random Coding Exponents and the Statistical Physics of Random Codes , 2007, IEEE Transactions on Information Theory.

[20]  R. Adler.  An introduction to continuity, extrema, and related topics for general Gaussian processes , 1990 .

[21]  Jacob Ziv,et al.  On the threshold effect in radar range estimation (Corresp.) , 1969, IEEE Trans. Inf. Theory.

[22]  B. Derrida.  Random-energy model: An exactly solvable model of disordered systems , 1981 .

[23]  Bernard Derrida,et al.  The random energy model , 1980 .

[24]  H. Nishimori.  Statistical Physics of Spin Glasses and Information Processing , 2001 .

[25]  Ehud Weinstein,et al.  A lower bound on the mean-square error in random parameter estimation , 1985, IEEE Trans. Inf. Theory.