Generalized Spacing-Statistics and a New Family of Non-Parametric Tests

Random divisions of an interval arise in various context, including statistics, physics, and geometric analysis. For testing the uniformity of a random partition of the unit interval $[0,1]$ into $k$ disjoint subintervals of size $(S_k[1],\ldots,S_k[k])$, Greenwood (1946) suggested using the squared $\ell_2$-norm of this size vector as a test statistic, prompting a number of subsequent studies. Despite much progress on understanding its power and asymptotic properties, attempts to find its exact distribution have succeeded so far for only small values of $k$. Here, we develop an efficient method to compute the distribution of the Greenwood statistic and more general spacing-statistics for an arbitrary value of $k$. Specifically, we consider random divisions of $\{1,2,\dots,n\}$ into $k$ subsets of consecutive integers and study $\|S_{n,k}\|^p_{p,w}$, the $p$th power of the weighted $\ell_p$-norm of the subset size vector $S_{n,k}=(S_{n,k}[1],\ldots,S_{n,k}[k])$ for arbitrary weights $w=(w_1,\ldots,w_k)$. We present an exact and quickly computable formula for its moments, as well as a simple algorithm to accurately reconstruct a probability distribution using the moment sequence. We also study various scaling limits, one of which corresponds to the Greenwood statistic in the case of $p=2$ and $w=(1,\ldots,1)$, and this connection allows us to obtain information about regularity, monotonicity and local behavior of its distribution. Lastly, we devise a new family of non-parametric tests using $\|S_{n,k}\|^p_{p,w}$ and demonstrate that they exhibit substantially improved power for a large class of alternatives, compared to existing popular methods such as the Kolmogorov-Smirnov, Cramer-von Mises, and Mann-Whitney/Wilcoxon rank-sum tests.