Higher Criticism in the Context of Unknown Distribution, Non-independence and Classification

Higher criticism has been proposed as a tool for highly multiple hypothesis testing or signal detection, initially in cases where the distribution of a test statistic (or the noise in a signal) is known and the component tests are statistically independent. In this paper we explore the extent to which the assumptions of known distribution and independence can be relaxed, and we consider too the application of higher criticism to classification. It is shown that effective distribution approximations can be achieved by using a threshold approach; that is, by disregarding data components unless their significance level exceeds a sufficiently high value. This method exploits the good relative accuracy of approximations to lighttailed distributions. In particular, it can be effective when the true distribution is founded on something like a Studentised mean, or on an average of related type, which is commonly the case in practice. The issue of dependence among vector components is also shown not to be a serious difficulty in many circumstances.