Conditional independence testing in Hilbert spaces with applications to functional data analysis

We study the problem of testing the null hypothesis that X and Y are conditionally independent given Z, where each of X, Y and Z may be functional random variables. This generalises, for example, testing the significance of X in a scalar-on-function linear regression model of response Y on functional regressors X and Z. We show however that even in the idealised setting where additionally (X,Y, Z) have a non-singular Gaussian distribution, the power of any test cannot exceed its size. Further modelling assumptions are needed to restrict the null and we argue that a convenient way of specifying these is based on choosing methods for regressing each of X and Y on Z. We thus propose as a test statistic, the Hilbert–Schmidt norm of the outer product of the resulting residuals, and prove that type I error control is guaranteed when the in-sample prediction errors are sufficiently small. We show this requirement is met by ridge regression in functional linear model settings without requiring any eigen-spacing conditions or lower bounds on the eigenvalues of the covariance of the functional regressor. We apply our test in constructing confidence intervals for truncation points in truncated functional linear models.

[1]  J. Robins,et al.  Semiparametric Efficiency in Multivariate Regression Models with Missing Data , 1995 .

[2]  R. Durrett Probability: Theory and Examples , 1993 .

[3]  Bernhard Schölkopf,et al.  Elements of Causal Inference: Foundations and Learning Algorithms , 2017 .

[4]  V. Koltchinskii,et al.  Oracle inequalities in empirical risk minimization and sparse recovery problems , 2011 .

[5]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[6]  Sang-Yun Oh,et al.  Partial Separability and Functional Graphical Models for Multivariate Gaussian Processes , 2021, Biometrika.

[7]  S. Wood On p-values for smooth components of an extended generalized additive model , 2013 .

[8]  C. Kraft Some conditions for consistency and uniform consistency of statistical procedures , 1955 .

[9]  Ana-Maria Staicu,et al.  Penalized function-on-function regression , 2015, Comput. Stat..

[10]  F. Scalora Abstract martingale convergence theorems , 1961 .

[11]  Hyejin Shin,et al.  Partial functional linear regression , 2009 .

[12]  E. Siebert Weak convergence of measures , 1984 .

[13]  G. Wahba,et al.  A Correspondence Between Bayesian Estimation on Stochastic Processes and Smoothing by Splines , 1970 .

[14]  J. Robins,et al.  Double/Debiased Machine Learning for Treatment and Structural Parameters , 2017 .

[15]  Halbert L. White, Jr.,et al.  CENTRAL LIMIT AND FUNCTIONAL CENTRAL LIMIT THEOREMS FOR HILBERT-VALUED DEPENDENT HETEROGENEOUS ARRAYS WITH APPLICATIONS , 1997, Econometric Theory.

[16]  N. Vakhania,et al.  Probability Distributions on Banach Spaces , 1987 .

[17]  Jonas Peters,et al.  Causal inference by using invariant prediction: identification and confidence intervals , 2015, 1501.01332.

[18]  P. Reiss,et al.  Functional Principal Component Regression and Functional Partial Least Squares , 2007 .

[19]  Gareth M. James,et al.  Functional additive regression , 2015, 1510.04064.

[20]  Philip T. Reiss,et al.  The International Journal of Biostatistics Fast Function-on-Scalar Regression with Penalized Basis Expansions , 2011 .

[21]  H. Holzmann,et al.  Uniform approximation in classical weak convergence theory. , 2019, 1903.09864.

[22]  Ciprian M Crainiceanu,et al.  Penalized Functional Regression , 2011, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[23]  Ana-Maria Staicu,et al.  Functional Additive Mixed Models , 2012, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[24]  Z. Q. John Lu,et al.  Nonparametric Functional Data Analysis: Theory And Practice , 2007, Technometrics.

[25]  Z. Bai,et al.  EFFECT OF HIGH DIMENSION: BY AN EXAMPLE OF A TWO SAMPLE PROBLEM , 1999 .

[26]  Caroline F Finch,et al.  Applications of functional data analysis: A systematic review , 2013, BMC Medical Research Methodology.

[27]  Hans-Georg Müller,et al.  Functional Data Analysis , 2016 .

[28]  R. Bass,et al.  Review: P. Billingsley, Convergence of probability measures , 1971 .

[29]  T. Tony Cai,et al.  Prediction in functional linear regression , 2006 .

[30]  Joel L. Horowitz,et al.  Methodology and convergence rates for functional linear regression , 2007, 0708.0466.

[31]  J. Florens,et al.  Functional linear regression with functional response , 2017 .

[32]  Maximilian Kasy,et al.  Uniformity and the Delta Method , 2015, Journal of Econometric Methods.

[33]  Fabian Scheipl,et al.  A general framework for functional regression modelling , 2017 .

[34]  T. Haavelmo,et al.  The probability approach in econometrics , 1944 .

[35]  Ciprian M. Crainiceanu,et al.  refund: Regression with Functional Data , 2013 .

[36]  Christophe Crambes,et al.  Asymptotics of prediction in functional linear regression with functional outputs , 2009, 0910.3070.

[37]  Bernhard Schölkopf,et al.  A Generalized Representer Theorem , 2001, COLT/EuroCOLT.

[38]  J. K. Hunter,et al.  Measure Theory , 2007 .

[39]  T. Hsing,et al.  Theoretical foundations of functional data analysis, with an introduction to linear operators , 2015 .

[40]  Jane-ling Wang Nonparametric Regression Analysis of Longitudinal Data , 2005 .

[41]  J. Robins,et al.  Adjusting for Nonignorable Drop-Out Using Semiparametric Nonresponse Models , 1999 .

[42]  M. Yuan,et al.  Model selection and estimation in the Gaussian graphical model , 2007 .

[43]  Aurore Delaigle,et al.  Methodology and theory for partial least squares applied to functional data , 2012, 1205.6367.

[44]  Giles Hooker,et al.  Truncated linear models for functional data , 2014, 1406.7732.

[45]  H. Müller,et al.  FUNCTIONAL RESPONSE MODELS , 2004 .

[46]  Frédéric Ferraty,et al.  Kernel regression with functional response , 2011 .

[47]  Jeffrey S. Morris Functional Regression , 2014, 1406.4068.

[48]  Arthur E. Hoerl,et al.  Ridge Regression: Biased Estimation for Nonorthogonal Problems , 2000, Technometrics.

[49]  Rajen Dinesh Shah,et al.  The hardness of conditional independence testing and the generalised covariance measure , 2018, The Annals of Statistics.

[50]  Jane-ling Wang,et al.  Functional linear regression analysis for longitudinal data , 2005, math/0603132.

[51]  David B. Dunson,et al.  Bayesian Graphical Models for Multivariate Functional Data , 2014, J. Mach. Learn. Res..

[52]  Nir Friedman,et al.  Probabilistic Graphical Models: Principles and Techniques - Adaptive Computation and Machine Learning , 2009 .

[53]  Xinghao Qiao,et al.  Functional Graphical Models , 2018, Journal of the American Statistical Association.

[54]  H. Müller,et al.  Functional quadratic regression , 2010 .

[55]  A. Vannucci,et al.  BICS Bath Institute for Complex Systems A note on time-dependent DiPerna-Majda measures , 2008 .

[56]  L. Lecam Convergence of Estimates Under Dimensionality Restrictions , 1973 .

[57]  Kellen Petersen August Real Analysis , 2009 .

[58]  Dudley,et al.  Real Analysis and Probability: Measurability: Borel Isomorphism and Analytic Sets , 2002 .

[59]  M. Yuan,et al.  A Reproducing Kernel Hilbert Space Approach to Functional Linear Regression , 2010, 1211.2607.