The Impossibility Region for Detecting Sparse Mixtures using the Higher Criticism

Abstract: Consider a multiple hypothesis testing setting involving rare/weak effects: relatively few tests, out of possibly many, deviate from their null hypothesis behavior. Summarizing the significance of each test by a P -value, we construct a global test against the null using the Higher Criticism (HC) statistics of these P-values. We calibrate the rare/weak model using parameters controlling the asymptotic distribution of non-null P -values near zero. We derive a region in the parameter space where the HC test is asymptotically powerless. Our derivation involves very different tools than previously used to show powerlessness of HC, relying on properties of the empirical processes underlying HC. In particular, our result applies to situations where HC is not asymptotically optimal, or when the asymptotically detectable region of the parameter space is unknown.