Canonical ensembles for potentially incompatible dependency networks with applications to medical data

A directed graph is either acyclic or cyclic. This paper focuses on the cyclic model, or dependency network, which represents a collection of univariate conditional distributions. The conditional approach allows a high level of flexibility in modeling because the dependency network is based on the notion that it is computationally convenient to estimate the local distribution of a variable given the remaining variables in a data set. However, the collection of conditional distributions individually estimated within a dependency network is generally not coherent with any joint distribution. The pseudo-Gibbs sampler (PGS) has often been used to estimate joint distributions for incompatible conditional models. We propose a new method for deriving a joint distribution from a given set of potentially incompatible univariate-conditional distributions such that the discrepancies between the given conditional distribution and those computed from the estimated joint distribution is minimized. The method is based on an ensemble of distributions, each of which can be derived from the canonical parameters of a set of given conditional distributions. Through simulation experiments and real data sets, we compare the performance of the ensemble method, the PGS, and a linear programming (LP)-based method. Our comparisons suggest that the ensemble method outperforms both the PGS and LP. The ensemble method is computationally efficient and scalable, and it therefore has the potential to open a new avenue for finding a nearly optimal solution for dependency networks of high dimensions.