Generalized p-Values in Significance Testing of Hypotheses in the Presence of Nuisance Parameters

Abstract This article examines some problems of significance testing for one-sided hypotheses of the form H 0 : θ ≤ θ 0 versus H 1 : θ > θ 0, where θ is the parameter of interest. In the usual setting, let x be the observed data and let T(X) be a test statistic such that the family of distributions of T(X) is stochastically increasing in θ. Define Cx as {X : T(X) — T(x) ≥ 0}. The p value is p(x) = sup θ≤θ0 Pr(X ∈ Cx | θ). In the presence of a nuisance parameter η, there may not exist a nontrivial Cx with a p value independent of η. We consider tests based on generalized extreme regions of the form Cx (θ, η) = {X : T(X; x, θ, η) ≥ T(x; x, θ, η)}, and conditions on T(X; x, θ, η) are given such that the p value p(x) = sup θ≤θ0 Pr(X ∈ Cx (θ, η)) is free of the nuisance parameter η, where T is stochastically increasing in θ. We provide a solution to the problem of testing hypotheses about the differences in means of two independent exponential distributions, a problem for which the fixed-level testing approach...