Asymptotically minimax regret for models with hidden variables

We study the problems of data compression, gambling and prediction of a string xn = x1x2...xn from an alphabet X, in terms of regret with respect to models with hidden variables including general mixture families. When the target class is a non-exponential family, a modification of Jeffreys prior which has measure outside the given family of densities was introduced to achieve the minimax regret [8], under certain regularity conditions. In this paper, we show that the models with hidden variables satisfy those regularity conditions, when the hidden variables' model is an exponential family. In paticular, we do not have to restrict the class of data strings so that the MLE is in the interior of the parameter space for the case of the general mixture family.

[1]  L. M. M.-T. Theory of Probability , 1929, Nature.

[2]  Miss A.O. Penney (b) , 1974, The New Yale Book of Quotations.

[3]  Jorma Rissanen,et al.  Fisher information and stochastic complexity , 1996, IEEE Trans. Inf. Theory.

[4]  Andrew R. Barron,et al.  Minimax redundancy for the class of memoryless sources , 1997, IEEE Trans. Inf. Theory.

[5]  Jorma Rissanen,et al.  The Minimum Description Length Principle in Coding and Modeling , 1998, IEEE Trans. Inf. Theory.

[6]  A. Barron,et al.  Asymptotically minimax regret by Bayes mixtures , 1998, Proceedings. 1998 IEEE International Symposium on Information Theory (Cat. No.98CH36252).

[7]  A. Barron,et al.  Mixture models achieving optimal coding regret , 1998, 1998 Information Theory Workshop (Cat. No.98EX131).

[8]  Andrew R. Barron,et al.  Asymptotic minimax regret for data compression, gambling, and prediction , 1997, IEEE Trans. Inf. Theory.

[9]  Shun-ichi Amari,et al.  Methods of information geometry , 2000 .

[10]  A. Barron,et al.  Asymptotically minimax regret for exponential families , 2005 .

[11]  A. Barron,et al.  STATISTICAL CURVATURE AND STOCHASTIC COMPLEXITY , 2006 .

[12]  Andrew R. Barron,et al.  Asymptotically minimax regret by Bayes mixtures for non-exponential families , 2013, 2013 IEEE Information Theory Workshop (ITW).

[13]  Tsutomu Kawabata,et al.  Properties of Jeffreys Mixture for Markov Sources , 2013, IEEE Transactions on Information Theory.

[14]  Gou Hosoya,et al.  国際会議参加報告:2014 IEEE International Symposium on Information Theory , 2014 .