A Bayesian Significance Test for Multinomial Distributions

SUMMARY A Bayesian significance test for multinomial distributions is discussed, together with the results of 18 numerical experiments in which the test is compared with non-Bayesian methods. Several different non-Bayesian criteria are considered because the circumstances under which their tail-area probabilities can be conveniently approximated differ from one to the other. A provisional empirical formula, connecting the Bayes factors with the tailarea probabilities, is found to be correct within a factor of 6 in the 18 experiments. As a by-product, a new non-Bayesian statistic is suggested by the theory, and its asymptotic distribution obtained. It seems to be useful sometimes when chi-squared is not, although chi-squared also has a Bayesian justification for large samples. The work originated in, and is relevant to, the problem of the estimation of multinomial probabilities, but significance tests are a better proving ground for assumptions concerning the initial (prior) distribution of the physical probabilities.

[1]  Harold Jeffreys,et al.  Further significance tests , 1936, Mathematical Proceedings of the Cambridge Philosophical Society.

[2]  J. Haldane.  THE EXACT VALUE OF THE MOMENTS OF THE DISTRIBUTION OF x2 USED AS A TEST OF GOODNESS OF FIT, WHEN EXPECTATIONS ARE SMALL , 1937 .

[3]  S. S. Wilks.  The Large-Sample Distribution of the Likelihood Ratio for Testing Composite Hypotheses , 1938 .

[4]  Maurice G. Kendall,et al.  Second Paper on Random Sampling Numbers , 1939 .

[5]  H. Jeffreys.  An invariant form for the prior probability in estimation problems , 1946, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[6]  Wilfred Perks,et al.  Some observations on inverse probability including a new indifference rule , 1947 .

[7]  P. Patnaik.  THE NON-CENTRAL χ2- AND F-DISTRIBUTIONS AND THEIR APPLICATIONS , 1949 .

[8]  L. J. Savage,et al.  Probability and the weighing of evidence , 1951 .

[9]  I. Good.  THE POPULATION FREQUENCIES OF SPECIES AND THE ESTIMATION OF POPULATION PARAMETERS , 1953 .

[10]  I. Good,et al.  THE NUMBER OF NEW SPECIES, AND THE INCREASE IN POPULATION COVERAGE, WHEN A SAMPLE IS INCREASED , 1956 .

[11]  C. Radhakrishna Rao,et al.  Some small sample tests of significance for a Poisson distribution , 1956 .

[12]  D. Lindley.  A STATISTICAL PARADOX , 1957 .

[13]  I. Good.  Saddle-point Methods for the Multinomial Distribution , 1957 .

[14]  I. Good.  Significance Tests in Parallel and in Series , 1958 .

[15]  I. Good.  THE MULTIVARIATE SADDLEPOINT METHOD AND CHI-SQUARED FOR THE MULTINOMIAL DISTRIBUTION , 1961 .

[16]  B. L. Welch,et al.  On Formulae for Confidence Points Based on Integrals of Weighted Likelihoods , 1963 .

[17]  M. Kendall,et al.  The advanced theory of statistics , 1945 .

[18]  D. Lindley.  The Bayesian Analysis of Contingency Tables , 1964 .

[19]  B. L. Welch.  On Comparisons between Confidence Point Procedures in the Case of a Single Parameter , 1965 .

[20]  H. O. Lancaster,et al.  SIZES OF THE χ2 TEST IN THE SYMMETRICAL MULTINOMIALS , 1965 .

[21]  D. Bartholomew,et al.  A comparison of some Bayesian and frequentist inferences. II. , 1965, Biometrika.

[22]  I. J. Good,et al.  How to Estimate Probabilities , 1966 .