Hypothesis Tests for Bernoulli Experiments: Ordering the Sample Space by Bayes Factors and Using Adaptive Significance Levels for Decisions

The main objective of this paper is to find the relation between the adaptive significance level presented here and the sample size. We statisticians know of the inconsistency, or paradox, in the current classical tests of significance that are based on p-value statistics that are compared to the canonical significance levels (10%, 5%, and 1%): “Raise the sample to reject the null hypothesis” is the recommendation of some ill-advised scientists! This paper will show that it is possible to eliminate this problem of significance tests. We present here the beginning of a larger research project. The intention is to extend its use to more complex applications such as survival analysis, reliability tests, and other areas. The main tools used here are the Bayes factor and the extended Neyman–Pearson Lemma.

[1]  J. Dickey.  The Weighted Likelihood Ratio, Linear Hypotheses on Normal Location Parameters , 1971 .

[2]  B. Greenberg,et al.  Gamma ventral capsulotomy for obsessive-compulsive disorder: a randomized clinical trial. , 2014, JAMA psychiatry.

[3]  H. Hoijtink,et al.  Error probabilities in default Bayesian hypothesis testing , 2016 .

[4]  D. Lindley.  A STATISTICAL PARADOX , 1957 .

[5]  Carlos Alberto de Bragança Pereira,et al.  Adaptative significance levels using optimal decision rules: Balancing by weighting the error probabilities , 2016 .

[6]  Carlos Alberto de Bragança Pereira,et al.  ON THE CONCEPT OF P-VALUE , 1988 .

[7]  D. Chakrabarty.  A New Bayesian Test to Test for the Intractability-Countering Hypothesis , 2013, 1304.5982.

[8]  M R Whittle,et al.  An unconditional exact test for the Hardy-Weinberg equilibrium law: sample-space ordering using the Bayes factor. , 2001, Genetics.

[9]  Carlos Alberto de Bragança Pereira,et al.  BAYESIAN HYPOTHESIS TEST: Using Surface Integrals to Distribute Prior Information Among the Hypotheses , 1995 .

[10]  Carlos Alberto de Bragança Pereira,et al.  Can a Significance Test Be Genuinely Bayesian , 2008 .

[11]  N. Lazar,et al.  The ASA Statement on p-Values: Context, Process, and Purpose , 2016 .

[12]  V. Johnson.  Revised standards for statistical evidence , 2013, Proceedings of the National Academy of Sciences.

[13]  Christian P Robert,et al.  Revised evidence for statistical standards , 2014, Proceedings of the National Academy of Sciences.

[14]  J. Ghosh.  Elimination of Nuisance Parameters , 1988 .

[15]  Dalmeet Singh Chawla.  Big names in statistics want to shake up much-maligned P value , 2017, Nature.

[16]  Julio Michael Stern,et al.  Bayesian evidence test for precise hypotheses , 2003 .

[17]  Michael Evans,et al.  Measuring statistical evidence using relative belief , 2015, Computational and structural biotechnology journal.

[18]  Roch Giorgi,et al.  Reproducibility issues in science, is P value really the only answer? , 2014, Proceedings of the National Academy of Sciences.

[19]  E. S. Pearson,et al.  On the Problem of the Most Efficient Tests of Statistical Hypotheses , 1933 .

[20]  David R. Cox,et al.  PRINCIPLES OF STATISTICAL INFERENCE , 2017 .

[21]  Luis Pericchi,et al.  Adaptive revised standards for statistical evidence , 2014, Proceedings of the National Academy of Sciences.

[22]  Walter L. Smith.  Probability and Statistics , 1959, Nature.

[23]  Carlos Alberto de Bragança Pereira,et al.  RELATIONSHIP BETWEEN BAYESIAN AND FREQUENTIST SIGNIFICANCE INDICES , 2012 .

[24]  J. Cornfield.  Sequential Trials, Sequential Analysis and the Likelihood Principle , 1966 .

[25]  Ming-Hui Chen,et al.  CALIBRATING BAYES FACTOR UNDER PRIOR PREDICTIVE DISTRIBUTIONS , 2005 .

[26]  M. Bartlett.  A comment on D. V. Lindley's statistical paradox , 1957 .

[27]  Steffen L. Lauritzen,et al.  Compatible prior distributions , 2000 .

[28]  L. M. M.-T..  Theory of Probability , 1929, Nature.

[29]  Christopher D. Chambers,et al.  Redefine statistical significance , 2017, Nature Human Behaviour.

[30]  Julio Michael Stern,et al.  Bayesian epistemic values: focus on surprise, measure probability! , 2014, Log. J. IGPL.