Generalized Schott type tests for complete independence in high dimensions

Abstract In the high dimensional setting, this article explores the problem of testing the complete independence of random variables having a multivariate normal distribution. A natural high-dimensional extension of the test in Schott (2005) is proposed for this purpose. The newly defined tests are asymptotically distribution-free as both the sample size and the number of variables go to infinity and hence have well-known critical values, accommodate situations where the number of variables is not small relative to the sample size and are applicable without specifying an explicit relationship between the number of variables and the sample size. In practice, as the true alternative hypothesis is unknown, it is unclear how to choose a powerful test. For this, we further propose an adaptive test that maintains high power across a wide range of situations. An extensive simulation study shows that the newly proposed tests are comparable to, and in many cases more powerful than, existing tests currently in the literature.

[1]  Fan Yang,et al.  Likelihood Ratio Tests for High‐Dimensional Normal Distributions , 2013, 1306.0254.

[2]  Wei Pan,et al.  An adaptive two-sample test for high-dimensional means , 2016, Biometrika.

[3]  Guangyu Mao.  Robust test for independence in high dimensions , 2017 .

[4]  Y. Qi,et al.  On Tests for Complete Independence of Normal Random Vectors , 2017, 1704.01673.

[5]  D. Najarzadeh.  Testing independence in high-dimensional multivariate normal data , 2019 .

[6]  P. Rosenberg,et al.  Pathway analysis by adaptive combination of P‐values , 2009, Genetic epidemiology.

[7]  H. Hotelling.  Relations Between Two Sets of Variates , 1936 .

[8]  Y. Qi,et al.  On Schott’s and Mao’s test statistics for independence of normal random vectors , 2018, Statistics & Probability Letters.

[9]  I. Johnstone.  On the distribution of the largest eigenvalue in principal components analysis , 2001 .

[10]  Jun Yu Li,et al.  Two Sample Tests for High Dimensional Covariance Matrices , 2012, 1206.0917.

[11]  Shu-rong Zheng,et al.  Testing the independence of sets of large-dimensional variables , 2013 .

[12]  Guangyu Mao.  A new test of independence for high-dimensional data , 2014 .

[13]  James R. Schott,et al.  Some tests for the equality of covariance matrices , 2001 .

[14]  Weidong Liu,et al.  Two‐sample test of high dimensional means under dependence , 2014 .

[15]  Guanghui Wang,et al.  A necessary test for complete independence in high dimensions using rank-correlations , 2013, J. Multivar. Anal..

[16]  Tiefeng Jiang,et al.  Likelihood ratio tests for covariance matrices of high-dimensional normal distributions , 2012 .

[17]  S. S. Wilks.  On the Independence of k Sets of Normally Distributed Statistical Variables , 1935 .

[18]  Yongcheng Qi,et al.  Likelihood Ratio Tests for High-Dimensional Normal Distributions , 2015 .

[19]  P. Hall,et al.  Martingale Limit Theory and Its Application , 1980 .

[20]  Guangyu Mao.  A note on testing complete independence for high dimensional data , 2015 .

[21]  R. Muirhead.  Aspects of Multivariate Statistical Theory , 1982, Wiley Series in Probability and Statistics.

[22]  D. F. Morrison,et al.  Multivariate Statistical Methods , 1968 .

[23]  James R. Schott,et al.  Testing for complete independence in high dimensions , 2005 .

[24]  Song-xi Chen,et al.  Tests for High-Dimensional Covariance Matrices , 2010, Random Matrices: Theory and Applications.

[25]  Tiefeng Jiang,et al.  The asymptotic distributions of the largest entries of sample correlation matrices , 2004, math/0406184.

[26]  W. Pan,et al.  ASYMPTOTICALLY INDEPENDENT U-STATISTICS IN HIGH-DIMENSIONAL TESTING. , 2018, Annals of statistics.

[27]  Guangyu Mao,et al.  Testing independence in high dimensions using Kendall's tau , 2018, Comput. Stat. Data Anal..

[28]  M. Srivastava.  Some Tests Concerning the Covariance Matrix in High Dimensional Data , 2005 .

[29]  Olivier Ledoit,et al.  Some hypothesis tests for the covariance matrix when the dimension is large compared to the sample size , 2002 .

[30]  Z. Bai,et al.  Corrections to LRT on large-dimensional covariance matrix by RMT , 2009, 0902.0552.

[31]  T. Cai,et al.  Two-Sample Covariance Matrix Testing and Support Recovery in High-Dimensional and Sparse Settings , 2013 .