A general duality approach to I-projections

Abstract I-projection problems arise in a myriad of situations and settings. In this paper, it is shown that under reasonable assumptions, a Fenchel type dual optimization problem exists which is equivalent to the stated I-projection problem for very general probability measures. This dual problem is often much more tractable than the original I-projection problem. I-projection problems which are equivalent to least square problems are also identified; primarily through the dual formulation. Several examples are examined to illustrate the duality structure and theorems.

[1]  I. N. Sanov.  On the probability of large deviations of random variables , 1958 .

[2]  I. Csiszár.  A geometric interpretation of Darroch and Ratcliff's generalized iterative scaling , 1989 .

[3]  R. Dykstra,et al.  Duality of I Projections and Maximum Likelihood Estimation for Log-Linear Models Under Cone Constraints , 1988 .

[4]  C. R. Rao,et al.  Linear Statistical Inference and its Applications , 1968 .

[5]  J. Ord,et al.  Characterization Problems in Mathematical Statistics , 1975 .

[6]  H. D. Brunk,et al.  The Isotonic Regression Problem and its Dual , 1972 .

[7]  E. Jaynes.  Information Theory and Statistical Mechanics , 1957 .

[8]  J. Darroch,et al.  Generalized Iterative Scaling for Log-Linear Models , 1972 .

[9]  I. Csiszár.  $I$-Divergence Geometry of Probability Distributions and Minimization Problems , 1975 .

[10]  F. T. Wright,et al.  Order restricted statistical inference , 1988 .

[11]  D. Luenberger.  Optimization by Vector Space Methods , 1968 .

[12]  V. Barbu,et al.  Convexity and optimization in banach spaces , 1972 .

[13]  H. D. Brunk.  Conditional Expectation Given A $\sigma$-Lattice and Applications , 1965 .

[14]  Solomon Kullback,et al.  Information Theory and Statistics , 1960 .

[15]  I. Good.  Maximum Entropy for Hypothesis Formulation, Especially for Multidimensional Contingency Tables , 1963 .

[16]  S. Kullback,et al.  Contingency tables with given marginals. , 1968, Biometrika.

[17]  David G. Luenberger,et al.  Linear and nonlinear programming , 1984 .