Linear transformation of binary random vectors and its application to approximating probability distributions

A nonsingular linear transformation of binary-valued random vectors y = xA which minimizes a mutual information criterion I(y) is considered. It is shown that a nonsingular A exists such that I(y) = 0 if and only if x has a generalized binomial distribution. Computational algorithms for seeking an optimal A are developed, and dimensionality reduction is discussed briefly. This linear transformation is useful in improving the approximation of probability distributions. Numerical examples are presented.