Integrating inconsistent data in a probabilistic model

In this paper we discuss the process of building a joint probability distribution from an input set of low-dimensional probability distributions. Since the solution of the problem for a consistent input set of probability distributions is known we concentrate on a setup where the input probability distributions are inconsistent. In this case the iterative proportional fitting procedure (IPFP), which converges in the consistent case, tends to come to cycles. We propose a new algorithm that converges even in inconsistent case. The important property of the algorithm is that it can be efficiently implemented exploiting decomposability of considered distributions.

[1]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[2]  E. Jaynes.  Information Theory and Statistical Mechanics , 1957 .

[3]  Petr Hájek,et al.  Uncertain information processing in expert systems , 1992 .

[4]  W. Deming,et al.  On a Least Squares Adjustment of a Sampled Frequency Table When the Expected Marginal Totals are Known , 1940 .

[5]  Steffen L. Lauritzen,et al.  Bayesian updating in causal probabilistic networks by local computations , 1990 .

[6]  H. Kellerer.  Verteilungsfunktionen mit gegebenen Marginalverteilungen , 1964 .

[7]  Radim Jiroušek,et al.  Inconsistent knowledge integration in a probabilistic model , 1995 .

[8]  N. Čencov.  Statistical Decision Rules and Optimal Inference , 2000 .

[9]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[10]  N. N. Vorob’ev.  Consistent Families of Measures and Their Extensions , 1962 .

[11]  Radim Jirousek,et al.  Solution of the marginal problem and decomposable distributions , 1991, Kybernetika.

[12]  Rodney W. Johnson,et al.  Comments on and correction to 'Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy' (Jan 80 26-37) , 1983, IEEE Trans. Inf. Theory.

[13]  I. Csiszár.  $I$-Divergence Geometry of Probability Distributions and Minimization Problems , 1975 .

[14]  M. Fréchet.  Sur les tableaux de correlation dont les marges sont donnees , 1951 .

[15]  Rodney W. Johnson,et al.  Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy , 1980, IEEE Trans. Inf. Theory.

[16]  S. Lauritzen.  The EM algorithm for graphical association models with missing data , 1995 .

[17]  Jeff B. Paris,et al.  A note on the inevitability of maximum entropy , 1990, Int. J. Approx. Reason..

[18]  R. Jirousek,et al.  On the effective implementation of the iterative proportional fitting procedure , 1995 .

[19]  S. Kullback,et al.  An Information-Theoretic Derivation of Certain Limit Relations for a Stationary Markov Chain , 1966 .

[20]  Daniel N. Osherson,et al.  Probability Bootstrapping: Improving Prediction by Fitting Extensional Models to Knowledgeable but Incoherent Probability Judgments , 1997 .

[21]  Huaiyu Zhu.  On Information and Sufficiency , 1997 .