Some Terminology and Notation in Information Theory

The main purpose of the paper is to stimulate thought concerning terminology, notation and exposition of some basic parts of information theory. The notation used here is intended to be simple, nearly self-explanatory, readable in words from left to right, and suggestive of new applications. Moreover, sufficient generality is preserved ensure that entropy can be interpreted without necessarily depending on the frequentist definition of probability (as a limiting frequency an infinite sequence of trials). Mention is made of some connections of the theory with inverse probability and with mathematical statistics in general.

[1]  D. Lindley.  On a Measure of the Information Provided by an Experiment , 1956 .

[2]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[3]  H. Jeffreys,et al.  Theory of probability , 1896 .

[4]  P. M. Woodward.  3 – INFORMATION THEORY , 1953 .

[5]  Fritz Emde,et al.  Funktionentafeln : mit Formfln [i.e. Formeln] und Kurven , 1933 .

[6]  Mark Bartlett,et al.  THE STATISTICAL SIGNIFICANCE OF ODD BITS OF INFORMATION , 1952 .

[7]  G. Barnard.  The Theory of Information , 1951 .

[8]  Harold Jeffreys,et al.  Further significance tests , 1936, Mathematical Proceedings of the Cambridge Philosophical Society.

[9]  I. Good.  THE POPULATION FREQUENCIES OF SPECIES AND THE ESTIMATION OF POPULATION PARAMETERS , 1953 .

[10]  Philip M. Woodward,et al.  Probability and Information Theory with Applications to Radar , 1954 .

[11]  W. J. McGill.  Multivariate information transmission , 1954, Trans. IRE Prof. Group Inf. Theory.

[12]  Fritz Emde,et al.  Funktionentafeln mit Formeln in Kurven , 2015 .

[13]  W. Ackermann,et al.  Grundzuge der Theoretischen Logik , 1928 .

[14]  L. J. Savage,et al.  Probability and the weighing of evidence , 1951 .

[15]  I. Good.  On the Estimation of Small Frequencies in Contingency Tables , 1956 .

[16]  G. L. S. Shackle,et al.  Uncertainty and Business Decisions. , 1957 .

[17]  M. Bartlett,et al.  The statistical approach to the analysis of time-series , 1953, Trans. IRE Prof. Group Inf. Theory.

[18]  B. McMillan.  The Basic Theorems of Information Theory , 1953 .

[19]  Stanford Goldman,et al.  Information theory , 1953 .