Introduction to Minimum Encoding Inference

This paper examines the minimumencoding approaches to inference, Minimum Message Length (MML) and Minimum Description Length (MDL). This paper was written with the objective of providing an introduction to this area for statisticians. We describe coding techniques for data, and examine how these techniques can be applied to perform inference and model selection.

[1]  J. Rissanen.  A UNIVERSAL PRIOR FOR INTEGERS AND ESTIMATION BY MINIMUM DESCRIPTION LENGTH , 1983 .

[2]  C. S. Wallace,et al.  A General Selection Criterion for Inductive Inference , 1984, ECAI.

[3]  Daniel S. Hirschberg,et al.  Data compression , 1987, CSUR.

[4]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..

[5]  Jonathan J. Oliver.  Decision Graphs - An Extension of Decision Trees , 1993 .

[6]  Ronald L. Rivest,et al.  Inferring Decision Trees Using the Minimum Description Length Principle , 1989, Inf. Comput..

[7]  C. S. Wallace,et al.  An Information Measure for Classification , 1968, Comput. J..

[8]  Rohan A. Baxter,et al.  MML and Bayesianism: similarities and differences: introduction to minimum encoding inference Part , 1994 .

[9]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[10]  D. Dowe,et al.  Model selection in linear regression using the MML criterion , 1994, DCC 1994.

[11]  Chris S. Wallace,et al.  A Program for Numerical Classification , 1970, Comput. J..

[12]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[13]  C. S. Wallace,et al.  Estimation and Inference by Compact Coding , 1987 .

[14]  Gregory J. Chaitin,et al.  Information-Theoretic Computational Complexity , 1974 .

[15]  C. S. Wallace,et al.  Classification by Minimum-Message-Length Inference , 1991, ICCI.

[16]  Jonathan J. Oliver,et al.  MDL and MML: Similarities and differences , 1994 .