Variance Reduction for Experiments with One-Sided Triggering using CUPED

In online experimentation, trigger-dilute analysis is an approach to obtain more precise estimates of intent-to-treat (ITT) effects when the intervention is only exposed, or "triggered", for a small subset of the population. Trigger-dilute analysis cannot be used for estimation when triggering is only partially observed. In this paper, we propose an unbiased ITT estimator with reduced variance for cases where triggeringstatus isonlyobserved in the treatmentgroup.Ourmethod is based on the efficiency augmentation idea of CUPED and draws upon identification frameworks from the principal stratification and instrumental variables literature.Theunbiasednessof our estimation approach relies on a testable assumption that an augmentation term used for covariate adjustment equals zero in expectation. When this augmentation termfailsamean-zero test,weshowhowourestimator can incorporate in-experiment observations to reduce the augmentation’s bias, by sacrificing the amount of variance reduced. This provides an explicit knob to trade offbiaswith variance.Wedemonstrate through simulations that our estimator can remain unbiased and achieve precision improvements as good as if triggering status were fully observed, and in somecases outperforms trigger-dilute analysis.