The role of duality in optimization problems involving entropy functionals with applications to information theory

We consider infinite-dimensional optimization problems involving entropy-type functionals in the objective function as well as as in the constraints. A duality theory is developed for such problems and applied to the reliability rate function problem in information theory.

[1]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[2]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[3]  S. Kullback,et al.  Information Theory and Statistics , 1959 .

[4]  R. Rockafellar.  Minimax Theorems and Conjugate Saddle-Functions. , 1964 .

[5]  Robert G. Gallager,et al.  A simple derivation of the coding theorem and some applications , 1965, IEEE Trans. Inf. Theory.

[6]  R. Rockafellar,et al.  Duality and stability in extremum problems involving convex functions. , 1967 .

[7]  R. Gallager.  Information Theory and Reliable Communication , 1968 .

[8]  A. Charnes,et al.  An extremal and information-theoretic characterization of some interzonal transfer models , 1972 .

[9]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[10]  A. Charnes,et al.  An Extremal Principle for Accounting Balance of a Resource Value-Transfer Economy: Existence, Uniqueness and Computation , 1974 .

[11]  Richard E. Blahut,et al.  Hypothesis testing and information theory , 1974, IEEE Trans. Inf. Theory.

[12]  A. Charnes,et al.  Constrained Kullback-Leibler Estimation; Generalized Cobb-Douglas Balance, and Unconstrained Convex Programming. , 1975 .

[13]  Suguru Arimoto.  Computation of random coding exponent functions , 1976, IEEE Trans. Inf. Theory.

[14]  A Charnes,et al.  A Dual Optimization Framework for Some Problems of Information Theory and Statistics. , 1977 .

[15]  Abraham Charnes,et al.  Constrained Information Theoretic Characterizations in Consumer Purchase Behaviour , 1978 .

[16]  J. P. Ponstein.  Approaches to the theory of optimization: List of symbols , 1980 .

[17]  Jacob Ponstein,et al.  Approaches to the Theory of Optimization , 1980 .

[18]  Toby Berger,et al.  Review of Information Theory: Coding Theorems for Discrete Memoryless Systems (Csiszár, I., and Körner, J.; 1981) , 1984, IEEE Trans. Inf. Theory.

[19]  R. Rockafellar.  Conjugate Duality and Optimization , 1987 .