Weak Convergence Analysis of Asymptotically Optimal Hypothesis Tests

In recent years, solutions to various hypothesis testing problems in the asymptotic setting have been proposed using the results from large deviation theory. Such tests are optimal in terms of appropriately defined error exponents. For the practitioner, however, error probabilities in the finite sample size setting are more important. In this paper, we show how results on weak convergence of the test statistic can be used to obtain better approximations for the error probabilities in the finite sample size setting. While this technique is popular among statisticians for common tests, we demonstrate its applicability for several recently proposed asymptotically optimal tests, including tests for robust goodness of fit, homogeneity tests, outlier hypothesis testing, and graphical model estimation.

[1]  S. Kh. Tumanyan,et al.  Asymptotic Distribution of The $\chi ^2 $ Criterion when the Number of Observations and Number of Groups Increase Simultaneously , 1956 .

[2]  G. P. Steck,et al.  Limit theorems for conditional distributions , 1957 .

[3]  Jayakrishnan Unnikrishnan,et al.  Asymptotically Optimal Matching of Multiple Sequences to Source Distributions and Training Sequences , 2014, IEEE Transactions on Information Theory.

[4]  Andrew R. Barron,et al.  Information-theoretic asymptotics of Bayes methods , 1990, IEEE Trans. Inf. Theory.

[5]  Amir Dembo,et al.  Large Deviations Techniques and Applications , 1998 .

[6]  Paul Valiant.  Testing symmetric properties of distributions , 2008, STOC '08.

[7]  Stephen E. Fienberg,et al.  Testing Statistical Hypotheses , 2005 .

[8]  Lang Tong,et al.  A Large-Deviation Analysis of the Maximum-Likelihood Learning of Markov Tree Structures , 2009, IEEE Transactions on Information Theory.

[9]  R. Stephenson.  A and V , 1962, The British journal of ophthalmology.

[10]  E. Lehmann.  Testing Statistical Hypotheses , 1960 .

[11]  Ilias Diakonikolas,et al.  Optimal Algorithms for Testing Closeness of Discrete Distributions , 2013, SODA.

[12]  Jayakrishnan Unnikrishnan,et al.  De-anonymizing private data by matching statistics , 2013, 2013 51st Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[13]  J. Oosterhoff,et al.  THE CHOICE OF CELLS IN CHI–SQUARE TESTS , 1985 .

[14]  Dana Ron,et al.  On Testing Expansion in Bounded-Degree Graphs , 2000, Studies in Complexity and Cryptography.

[15]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[16]  Sean P. Meyn,et al.  Worst-case large-deviation asymptotics with application to queueing and information theory , 2006 .

[17]  W. Hoeffding.  Asymptotically Optimal Tests for Multinomial Distributions , 1965 .

[18]  Carl N. Morris,et al.  CENTRAL LIMIT THEOREMS FOR MULTINOMIAL SUMS , 1975 .

[19]  Imre Csiszár,et al.  Information Theory and Statistics: A Tutorial , 2004, Found. Trends Commun. Inf. Theory.

[20]  Ronitt Rubinfeld,et al.  Testing random variables for independence and identity , 2001, Proceedings 2001 IEEE International Conference on Cluster Computing.

[21]  Sean P. Meyn,et al.  Generalized Error Exponents for Small Sample Universal Hypothesis Testing , 2012, IEEE Transactions on Information Theory.

[22]  A. Wald,et al.  On the Choice of the Number of Class Intervals in the Application of the Chi Square Test , 1942 .

[23]  I JordanMichael,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008 .

[24]  Alon Orlitsky,et al.  Sublinear algorithms for outlier detection and generalized closeness testing , 2014, 2014 IEEE International Symposium on Information Theory.

[25]  Sean P. Meyn,et al.  On thresholds for robust goodness-of-fit tests , 2010, 2010 IEEE Information Theory Workshop.

[26]  James E. Smith,et al.  Generalized Chebychev Inequalities: Theory and Applications in Decision Analysis , 1995, Oper. Res..

[27]  A. V. D. Vaart,et al.  Asymptotic Statistics: Frontmatter , 1998 .

[28]  Alon Orlitsky,et al.  Competitive Closeness Testing , 2011, COLT.

[29]  R. R. Bahadur,et al.  On Deviations of the Sample Mean , 1960 .

[30]  Sean P. Meyn,et al.  Classification with high-dimensional sparse samples , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[31]  Jayakrishnan Unnikrishnan.  Model-fitting in the presence of outliers , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[32]  Sean P. Meyn,et al.  Universal and Composite Hypothesis Testing via Mismatched Divergence , 2009, IEEE Transactions on Information Theory.

[33]  Martin Vetterli,et al.  Where You Are Is Who You Are: User Identification by Matching Statistics , 2015, IEEE Transactions on Information Forensics and Security.

[34]  M. A. Johnson,et al.  An investigation of phase-distribution moment-matching algorithms for use in queueing models , 1991, Queueing Syst. Theory Appl..

[35]  William Cyrus Navidi,et al.  Statistics for Engineers and Scientists , 2004 .

[36]  D. Luenberger.  Optimization by Vector Space Methods , 1968 .

[37]  Yu. I. Medvedev.  Separable Statistics in a Polynomial Scheme. I , 1977 .

[38]  Ronitt Rubinfeld,et al.  Testing Closeness of Discrete Distributions , 2010, JACM.

[39]  M. Quine,et al.  Normal Approximations to Sums of Scores Based on Occupancy Numbers , 1984 .

[40]  Jayakrishnan Unnikrishnan.  On optimal two sample homogeneity tests for finite alphabets , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[41]  P. Halmos,et al.  Berry-Esseen bounds for the multi-dimensional central limit theorem , 1968 .

[42]  Pramod Viswanath,et al.  Classification of Homogeneous Data With Large Alphabets , 2013, IEEE Transactions on Information Theory.

[43]  Sirin Nitinawarat,et al.  Universal outlier hypothesis testing , 2013, 2013 IEEE International Symposium on Information Theory.

[44]  Ofer Shayevitz,et al.  On Rényi measures and hypothesis testing , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[45]  Michael Gutman,et al.  Asymptotically optimal classification for multiple tests with empirically observed statistics , 1989, IEEE Trans. Inf. Theory.

[46]  R. Bass,et al.  Review: P. Billingsley, Convergence of probability measures , 1971 .

[47]  S. S. Wilks.  The Large-Sample Distribution of the Likelihood Ratio for Testing Composite Hypotheses , 1938 .

[48]  A. Barron.  Uniformly Powerful Goodness of Fit Tests , 1989 .

[49]  Lars Holst,et al.  Asymptotic normality and efficiency for certain goodness-of-fit tests , 1972 .

[50]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[51]  Liam Paninski,et al.  A Coincidence-Based Test for Uniformity Given Very Sparsely Sampled Discrete Data , 2008, IEEE Transactions on Information Theory.

[52]  Alon Orlitsky,et al.  Universal compression of memoryless sources over unknown alphabets , 2004, IEEE Transactions on Information Theory.