A Note on the Applied Use of MDL Approximations

An applied problem is discussed in which two nested psychological models of retention are compared using minimum description length (MDL). The standard Fisher information approximation to the normalized maximum likelihood is calculated for these two models, with the result that the full model is assigned a smaller complexity, even for moderately large samples. A geometric interpretation for this behavior is considered, along with its practical implications.

[1]  Yong Su.  3 Minimum Description Length and Cognitive Modeling , 2003 .

[2]  Christian P. Robert,et al.  Monte Carlo Statistical Methods (Springer Texts in Statistics) , 2005 .

[3]  M. Schervish.  Theory of Statistics , 1995 .

[4]  I. J. Myung,et al.  Counting probability distributions: Differential geometry and model selection , 2000, Proc. Natl. Acad. Sci. USA.

[5]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[6]  Jorma Rissanen,et al.  Strong optimality of the normalized ML models as universal codes and information in data , 2001, IEEE Trans. Inf. Theory.

[7]  Mark A. Pitt,et al.  Advances in Minimum Description Length: Theory and Applications , 2005 .

[8]  Vijay Balasubramanian,et al.  Statistical Inference, Occam's Razor, and Statistical Mechanics on the Space of Probability Distributions , 1996, Neural Computation.

[9]  Jay I. Myung,et al.  Assessing the distinguishability of models and the informativeness of data , 2004, Cognitive Psychology.

[10]  Peter Gr Unwald.  The minimum description length principle and reasoning under uncertainty , 1998 .

[11]  Jorma Rissanen,et al.  Fisher information and stochastic complexity , 1996, IEEE Trans. Inf. Theory.

[12]  Christian P. Robert,et al.  Monte Carlo Statistical Methods , 2005, Springer Texts in Statistics.

[13]  D. Rubin,et al.  One Hundred Years of Forgetting : A Quantitative Description of Retention , 1996 .

[14]  Michael D. Lee,et al.  An application of minimum description length clustering to partitioning learning curves , 2005, Proceedings. International Symposium on Information Theory, 2005. ISIT 2005..

[15]  Wayne A. Wickelgren,et al.  Trace resistance and the decay of long-term memory. , 1972 .