Significance Tests in Parallel and in Series

Abstract The advice is often given that significance tests should be selected before sampling evidence is examined. It is suggested here that this advice is appropriate only for inexperienced statisticians, and an approximate rule of thumb is tentatively proposed in the hope of provoking discussion, namely that the statistician could in some cases use a harmonic mean or weighted harmonic mean of the tail-area probabilities arising from various tests, all on the same evidence (tests in “parallel”). This rule of thumb should not be used if the statistician can think of anything better to do, and especially of course if he is in a position to use a sufficient statistic (or an “efficacious” one, in a sense defined below). An application is given to the judgment of the weights that may be used for combining tests in series.

[1]  Karl Pearson,et al.  ON THE PROBABILITY THAT TWO INDEPENDENT DISTRIBUTIONS OF FREQUENCY ARE REALLY SAMPLES FROM THE SAME PARENT POPULATION , 1932 .

[2]  Karl Pearson,et al.  FURTHER APPLICATIONS IN STATISTICS OF THE Tm (x) BESSEL FUNCTION , 1932 .

[3]  Note on a Memoir by A. E. R. Church , 1932 .

[4]  W. Allen Wallis,et al.  Compounding Probabilities from Independent Significance Tests , 1942 .

[5]  Frank Sandon,et al.  The Advanced Theory of Statistics. II , 1947, The Mathematical Gazette.

[6]  J. Haldane,et al.  A new estimate of the linkage between the genes for colourblindness and haemophilia in man. , 1947, Annals of eugenics.

[7]  H. Robbins,et al.  Application of the Method of Mixtures to Quadratic Forms in Normal Variates , 1949 .

[8]  L. J. Savage,et al.  Probability and the weighing of evidence , 1951 .

[9]  D. Darling.  On a Class of Problems Related to the Random Division of an Interval , 1953 .

[10]  G. Box.  Some Theorems on Quadratic Forms Applied in the Study of Analysis of Variance Problems, I. Effect of Inequality of Variance in the One-Way Classification , 1954 .

[11]  W. G. Cochran.  Some Methods for Strengthening the Common χ 2 Tests , 1954 .

[12]  I. J. Good,et al.  On the Weighted Combination of Significance Tests , 1955 .

[13]  J. Gurland,et al.  Distribution of Definite and of Indefinite Quadratic Forms , 1955 .

[14]  Correlated Random Normal Deviates , 1956 .

[15]  I. J. Good,et al.  Which Comes First, Probability or Statistics? , 1956 .

[16]  I. Good.  Saddle-point Methods for the Multinomial Distribution , 1957 .