Type Classes of Tree Models

It is well known that a tree model does not always admit a finite-state machine (FSM) representation with the same (minimal) number of parameters. Therefore, known characterizations of type classes for FSMs do not apply, in general, to tree models. In this paper, the type class of a string with respect to a tree model is studied, and an exact formula is derived for the size of the class. The formula, which applies to arbitrary trees, generalizes Whittle's formula for FSMs. The derivation is more intricate than the FSM case, since some basic properties of FSM types do not hold in general for tree-model types. The derivation also yields an efficient enumeration of the tree-model type class, which has applications in universal data compression and universal simulation. A formula for the number of type classes with respect to a given tree is also derived. The formula is asymptotically tight up to multiplication by a constant and also generalizes a known result for FSMs.

[1]  Jorma Rissanen,et al.  Complexity of strings in the class of Markov sources , 1986, IEEE Trans. Inf. Theory.

[2]  Giuseppe Longo,et al.  The error exponent for the noiseless encoding of finite ergodic Markov sources , 1981, IEEE Trans. Inf. Theory.

[3]  P. Whittle,et al.  Some Distribution and Moment Formulae for the Markov Chain , 1955 .

[4]  Gadiel Seroussi,et al.  On universal types , 2004, IEEE Transactions on Information Theory.

[5]  L. Goddard.  Information Theory , 1962, Nature.

[6]  Neri Merhav,et al.  On universal simulation of information sources using training data , 2004, IEEE Transactions on Information Theory.

[7]  Neri Merhav,et al.  Optimal sequential probability assignment for individual sequences , 1994, IEEE Trans. Inf. Theory.

[8]  P. Billingsley,et al.  Statistical Methods in Markov Chains , 1961 .

[9]  Philippe Jacquet,et al.  Markov types and minimax redundancy for Markov sources , 2004, IEEE Transactions on Information Theory.

[10]  Byoung-Seon Choi,et al.  Conditional limit theorems under Markov conditioning , 1987, IEEE Trans. Inf. Theory.

[11]  Meir Feder,et al.  A universal finite memory source , 1995, IEEE Trans. Inf. Theory.

[12]  de Ng Dick Bruijn,et al.  Circuits and Trees in Oriented Linear Graphs , 1951 .

[13]  L. A. Goodman.  Exact Probabilities and Asymptotic Relationships for Some Statistics from $m$-th Order Markov Chains , 1958 .

[14]  O. F. Cook.  The Method of Types , 1898 .

[15]  P. Bühlmann,et al.  Variable Length Markov Chains: Methodology, Computing, and Software , 2004 .

[16]  Frans M. J. Willems,et al.  The context-tree weighting method: basic properties , 1995, IEEE Trans. Inf. Theory.

[17]  Imre Csisźar,et al.  The Method of Types , 1998, IEEE Trans. Inf. Theory.

[18]  Gadiel Seroussi,et al.  Linear time universal coding and time reversal of tree sources via FSM closure , 2004, IEEE Transactions on Information Theory.