Extremal distributions in information theory and hypothesis testing

Many problems in information theory can be distilled to an optimization problem over a space of probability distributions. The most important examples are in communication theory, where it is necessary to maximize mutual information in order to compute channel capacity, and the classical hypothesis testing problem in which an optimal test is based on the maximization of divergence. Two general classes of optimization problems are considered in this paper: convex and linear programs, where the constraint set is defined by a finite number of moment constraints.

[1]  Venugopal V. Veeravalli,et al.  Decentralized detection in sensor networks , 2003, IEEE Trans. Signal Process..

[2]  I. Olkin,et al.  Inequalities: Theory of Majorization and Its Applications , 1980 .

[3]  J. G. Gander,et al.  An introduction to signal detection and estimation , 1990 .

[4]  A. Viterbi.  CDMA: Principles of Spread Spectrum Communication , 1995 .

[5]  W. J. Studden,et al.  Tchebycheff Systems: With Applications in Analysis and Statistics. , 1967 .

[6]  H. Vincent Poor,et al.  The noncoherent rician fading Channel-part I: structure of the capacity-achieving input , 2005, IEEE Transactions on Wireless Communications.

[7]  W. Whitt.  Bivariate Distributions with Given Marginals , 1976 .

[8]  Thomas L. Marzetta,et al.  Capacity of a Mobile Multiple-Antenna Communication Link in Rayleigh Flat Fading , 1999, IEEE Trans. Inf. Theory.

[9]  R. Gallager.  Power Limited Channels: Coding, Multiaccess, and Spread Spectrum , 2002 .

[10]  Shlomo Shamai,et al.  On the capacity-achieving distribution of the discrete-time non-coherent additive white Gaussian noise channel , 2002, Proceedings IEEE International Symposium on Information Theory,.

[11]  P. J. Huber.  A Robust Version of the Probability Ratio Test , 1965 .

[12]  W. Hoeffding.  Probability Inequalities for sums of Bounded Random Variables , 1963 .

[13]  Leonard Eugene Dickson.  An invariantive investigation of irreducible binary modular forms , 1911 .

[14]  H. Vincent Poor,et al.  Minimax robust decentralized detection , 1994, IEEE Trans. Inf. Theory.

[15]  Shlomo Shamai,et al.  The capacity of average and peak-power-limited quadrature Gaussian channels , 1995, IEEE Trans. Inf. Theory.

[16]  Ioana Popescu,et al.  Optimal Inequalities in Probability Theory: A Convex Optimization Approach , 2005, SIAM J. Optim..

[17]  G. Bennett.  Probability Inequalities for the Sum of Independent Random Variables , 1962 .

[18]  Shlomo Shamai,et al.  Worst-case power-constrained noise for binary-input channels , 1992, IEEE Trans. Inf. Theory.

[19]  M. A. Johnson,et al.  Matching moments to phase distributions: nonlinear programming approaches , 1990 .

[20]  Sean P. Meyn,et al.  Asymptotic robust Neyman-Pearson hypothesis testing based on moment classes , 2004, International Symposium onInformation Theory, 2004. ISIT 2004. Proceedings..

[21]  Charuhas Pandit.  Robust Statistical Modeling Based on Moment Classes, With Applications to Admission Control, Large Deviations and Hypothesis Testing , 2004 .

[22]  Richard E. Blahut,et al.  Hypothesis testing and information theory , 1974, IEEE Trans. Inf. Theory.

[23]  D. Bertsimas,et al.  Moment Problems and Semidefinite Optimization , 2000 .

[24]  Sean P. Meyn,et al.  Characterization and computation of optimal distributions for channel coding , 2005, IEEE Transactions on Information Theory.

[25]  H. Vincent Poor,et al.  An Introduction to Signal Detection and Estimation , 1994, Springer Texts in Electrical Engineering.

[26]  Joel G. Smith,et al.  The Information Capacity of Amplitude- and Variance-Constrained Scalar Gaussian Channels , 1971, Inf. Control..

[27]  P. J. Huber,et al.  Minimax Tests and the Neyman-Pearson Lemma for Capacities , 1973 .

[28]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[29]  Sean P. Meyn,et al.  Extremal Distributions and Worst-Case Large-Deviation Bounds , 2004 .

[30]  W. Hoeffding.  Asymptotically Optimal Tests for Multinomial Distributions , 1965 .

[31]  Ibrahim C. Abou-Faycal,et al.  The capacity of discrete-time memoryless Rayleigh-fading channels , 2001, IEEE Trans. Inf. Theory.

[32]  P. Diaconis.  Application of the method of moments in probability and statistics , 1987 .

[33]  Sean P. Meyn,et al.  Robust Measurement-Based Admission Control Using Markov's Theory of Canonical Distributions , 2006, IEEE Transactions on Information Theory.

[34]  M. A. Johnson,et al.  An investigation of phase-distribution moment-matching algorithms for use in queueing models , 1991, Queueing Syst. Theory Appl..

[35]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[36]  Y. Tong,et al.  Convex Functions, Partial Orderings, and Statistical Applications , 1992 .