MDL/Bayesian Criteria Based on Universal Coding/Measure

In the minimum description length (MDL) and Bayesian criteria, we construct description length of data z n = z 1 ⋯ z n of length n such that the length divided by n almost converges to its entropy rate as n → ∞, assuming z i is in a finite set A. In model selection, if we knew the true probability P of z n ∈ A n , we would choose a model F such that the posterior probability of F given z n is maximized. But, in many situations, we use Q:A n → [0,1] such that \(\sum_{z^n\in A^n}Q(z^n)\leq 1\) rather than P because only data z n are available. In this paper, we consider an extension such that each of the attributes in data can be either discrete or continuous. The main issue is what Q is qualified to be an alternative to P in the generalized situations. We propose the condition in terms of the Radon-Nikodym derivative of P with respect to Q, and give the procedure of constructing Q in the general setting. As a result, we obtain the MDL/Bayesian criteria in a general sense.