Sequential causal inference in a single world of connected units

We consider adaptive designs for a trial involving N individuals that we follow along T time steps. We allow for the variables of one individual to depend on its past and on the past of other individuals. Our goal is to learn a mean outcome, averaged across the N individuals, that we would observe, if we started from some given initial state, and we carried out a given sequence of counterfactual interventions for τ time steps. We show how to identify a statistical parameter that equals this mean counterfactual outcome, and how to perform inference for this parameter, while adaptively learning an oracle design defined as a parameter of the true data generating distribution. Oracle designs of interest include the design that maximizes the efficiency for a statistical parameter of interest, or designs that mix the optimal treatment rule with a certain exploration distribution. We also show how to design adaptive stopping rules for sequential hypothesis testing. This setting presents unique technical challenges. Unlike in usual statistical settings where the data consists of several independent observations, here, due to network and temporal dependence, the data reduces to one single observation with dependent components. In particular, this precludes the use of sample splitting techniques. We therefore had to develop a new equicontinuity result and guarantees for estimators fitted on dependent data. Furthermore, since we want to design an adaptive stopping rule, we need guarantees over the joint distribution of the sequence of estimators. In particular, this requires our equicontinuity result to hold almost surely, and our convergence guarantees on nuisance estimators to hold uniformly in time. We introduce a nonparametric class of functions, which we argue is a realistic statistical model for nuisance parameters, and is such that we can check the required equicontinuity condition and show uniform-in-time convergence guarantees. We were motivated to work on this problem by the following two questions. (1) In the context of a sequential adaptive trial with K treatment arms, how to design a procedure to identify in as few rounds as possible the treatment arm with best final outcome? (2) In the context of sequential randomized disease testing at the scale of a city, how to estimate and infer the value of an optimal testing and isolation strategy?

[1]  J Mark,et al.  The Construction and Analysis of Adaptive Group Sequential Designs , 2008 .

[2]  Chris A. J. Klaassen,et al.  Consistent Estimation of the Influence Function of Locally Asymptotically Linear Estimators , 1987 .

[3]  Ramon van Handel.  On the minimal penalty for Markov order estimation , 2009, ArXiv.

[4]  M. Hudgens,et al.  Toward Causal Inference With Interference , 2008, Journal of the American Statistical Association.

[5]  M. J. van der Laan,et al.  The International Journal of Biostatistics Targeted Maximum Likelihood Learning , 2011 .

[6]  Richard C. Bradley,et al.  Introduction to strong mixing conditions , 2007 .

[7]  Edoardo M. Airoldi,et al.  Model-assisted design of experiments in the presence of network-correlated outcomes , 2015, Biometrika.

[8]  Antoine Chambaz,et al.  Generalized Policy Elimination: an efficient algorithm for Nonparametric Contextual Bandits , 2020, UAI.

[9]  M. J. Laan,et al.  Online targeted learning for time series , 2018 .

[10]  Aurélien F. Bibaut,et al.  Fast rates for empirical risk minimization over c\`adl\`ag functions with bounded sectional variation norm , 2019 .

[11]  Michael I. Jordan,et al.  Convexity, Classification, and Risk Bounds , 2006 .

[12]  Jasjeet S. Sekhon,et al.  Time-uniform, nonparametric, nonasymptotic confidence sequences , 2020, The Annals of Statistics.

[13]  D. McLeish.  Dependent Central Limit Theorems and Invariance Principles , 1974 .

[14]  Tyler J VanderWeele,et al.  On causal inference in the presence of interference , 2012, Statistical methods in medical research.

[15]  Elizabeth L. Ogburn,et al.  Causal Inference for Social Network Data , 2017, Journal of the American Statistical Association.

[16]  R. Khan,et al.  Sequential Tests of Statistical Hypotheses. , 1972 .

[17]  S. Murphy,et al.  Assessing Time-Varying Causal Effect Moderation in Mobile Health , 2016, Journal of the American Statistical Association.

[18]  Masatoshi Uehara,et al.  Efficiently Breaking the Curse of Horizon in Off-Policy Evaluation with Double Reinforcement Learning , 2019 .

[19]  H. Robbins.  Some aspects of the sequential design of experiments , 1952 .

[20]  Adityanand Guntuboyina,et al.  Multivariate extensions of isotonic regression and total variation denoising via entire monotonicity and Hardy–Krause variation , 2019, 1903.01395.

[21]  Susan Gruber,et al.  One-Step Targeted Minimum Loss-based Estimation Based on Universal Least Favorable One-Dimensional Submodels , 2016, The international journal of biostatistics.

[22]  M. J. Laan,et al.  Targeted Learning: Causal Inference for Observational and Experimental Data , 2011 .

[23]  R. C. Bradley.  Basic properties of strong mixing conditions. A survey and some open questions , 2005, math/0511078.

[24]  Masatoshi Uehara,et al.  Double Reinforcement Learning for Efficient Off-Policy Evaluation in Markov Decision Processes , 2019, J. Mach. Learn. Res..

[25]  Mark J. van der Laan,et al.  Causal Inference for a Population of Causally Connected Units , 2014, Journal of causal inference.

[26]  E. Rio,et al.  Bernstein inequality and moderate deviations under strong mixing conditions , 2012, 1202.4777.

[27]  Soumendu Sundar Mukherjee,et al.  Weak convergence and empirical processes , 2019 .

[28]  Mark J. van der Laan,et al.  The Highly Adaptive Lasso Estimator , 2016, 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA).

[29]  Guillaume W. Basse,et al.  Randomization tests of causal effects under interference , 2019, Biometrika.

[30]  M. J. van der Laan.  A Generally Efficient Targeted Minimum Loss Based Estimator based on the Highly Adaptive Lasso , 2017, The international journal of biostatistics.