Split-Treatment Analysis to Rank Heterogeneous Causal Effects for Prospective Interventions

For many kinds of interventions, such as a new advertisement, marketing intervention, or feature recommendation, it is important to target a specific subset of people for maximizing its benefits at minimum cost or potential harm. However, a key challenge is that no data is available about the effect of such a prospective intervention since it has not been deployed yet. In this work, we propose a split-treatment analysis that ranks the individuals most likely to be positively affected by a prospective intervention using past observational data. Unlike standard causal inference methods, the split-treatment method does not need any observations of the target treatments themselves. Instead it relies on observations of a proxy treatment that is caused by the target treatment. Under reasonable assumptions, we show that the ranking of heterogeneous causal effect based on the proxy treatment is the same as the ranking based on the target treatment's effect. In the absence of any interventional data for cross-validation, Split-Treatment uses sensitivity analyses for unobserved confounding to select model parameters. We apply Split-Treatment to both a simulated data and a large-scale, real-world targeting task and validate our discovered rankings via a randomized experiment for the latter.

[1]  Scott Counts,et al.  Using Longitudinal Social Media Analysis to Understand the Effects of Early College Alcohol Use , 2018, ICWSM.

[2]  Uri Shalit,et al.  Estimating individual treatment effect: generalization bounds and algorithms , 2016, ICML.

[3]  Illtyd Trethowan Causality , 1938 .

[4]  Duncan J. Watts,et al.  Split-door criterion: Identification of causal effects through auxiliary outcomes , 2016, The Annals of Applied Statistics.

[5]  Liang Tang,et al.  Automatic ad format selection via contextual bandits , 2013, CIKM.

[6]  Ang Li,et al.  Unit Selection Based on Counterfactual Logic , 2019, IJCAI.

[7]  J. Robins,et al.  Sensitivity Analysis for Selection bias and unmeasured Confounding in missing Data and Causal inference models , 2000 .

[8]  J. Pearl Causality: Models, Reasoning and Inference , 2000 .

[9]  Taghi M. Khoshgoftaar,et al.  A Survey of Collaborative Filtering Techniques , 2009, Adv. Artif. Intell..

[10]  James M. Robins,et al.  Marginal Structural Models versus Structural nested Models as Tools for Causal inference , 2000 .

[11]  Joshua D. Angrist,et al.  Identification of Causal Effects Using Instrumental Variables , 1993 .

[12]  J. Angrist,et al.  Two-Stage Least Squares Estimation of Average Causal Effects in Models with Variable Treatment Intensity , 1995 .

[13]  Ran Gilad-Bachrach,et al.  DART: Dropouts meet Multiple Additive Regression Trees , 2015, AISTATS.

[14]  David H. Reiley,et al.  Online ads and offline sales: measuring the effect of retail advertising via a controlled experiment on Yahoo! , 2014 .

[15]  John Langford,et al.  Off-policy evaluation for slate recommendation , 2016, NIPS.

[16]  Susan Athey,et al.  The State of Applied Econometrics - Causality and Policy Evaluation , 2016, 1607.00699.

[17]  Amit Sharma,et al.  DoWhy: An End-to-End Library for Causal Inference , 2020, ArXiv.

[18]  J. Pearl Causal diagrams for empirical research , 1995 .

[19]  P. Rosenbaum Design of Observational Studies , 2009, Springer Series in Statistics.

[20]  S. Sundar,et al.  Does Web Advertising Work? Memory for Print vs. Online Media , 1998 .

[21]  Marc L. Resnick,et al.  Ergonomics Society Annual Meeting , 2013 .

[22]  Zhiwei Steven Wu,et al.  Orthogonal Random Forest for Causal Inference , 2018, ICML.

[23]  Christian Hansen,et al.  Double/Debiased/Neyman Machine Learning of Treatment Effects , 2017, 1701.08687.

[24]  Brett R. Gordon,et al.  A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook , 2018, Mark. Sci..

[25]  Duncan J. Watts,et al.  Estimating the Causal Impact of Recommendation Systems from Observational Data , 2015, EC.

[26]  Jian Yang,et al.  Robust Tree-based Causal Inference for Complex Ad Effectiveness Analysis , 2015, WSDM.

[27]  Tor Lattimore,et al.  Causal Bandits: Learning Good Interventions via Causal Inference , 2016, NIPS.

[28]  Thomas Nedelec,et al.  Offline A/B Testing for Recommender Systems , 2018, WSDM.

[29]  Thorsten Joachims,et al.  Recommendations as Treatments: Debiasing Learning and Evaluation , 2016, ICML.

[30]  R. Winer A Framework for Customer Relationship Management , 2001 .

[31]  Sören R. Künzel,et al.  Metalearners for estimating heterogeneous treatment effects using machine learning , 2017, Proceedings of the National Academy of Sciences.

[32]  Steven L. Scott,et al.  Inferring causal impact using Bayesian structural time-series models , 2015, 1506.00356.

[33]  Elizabeth A Stuart,et al.  Matching methods for causal inference: A review and a look forward. , 2010, Statistical science : a review journal of the Institute of Mathematical Statistics.

[34]  Mark J van der Laan,et al.  Sensitivity Analysis for Causal Inference under Unmeasured Confounding and Measurement Error Problems , 2013, The international journal of biostatistics.

[35]  Rong Ge,et al.  Evaluating online ad campaigns in a pipeline: causal models at scale , 2010, KDD.

[36]  Wei Chu,et al.  A contextual-bandit approach to personalized news article recommendation , 2010, WWW '10.

[37]  Sinan Aral,et al.  Targeting for long-term outcomes , 2020, ArXiv.

[38]  Stefan Wager,et al.  Estimation and Inference of Heterogeneous Treatment Effects using Random Forests , 2015, Journal of the American Statistical Association.

[39]  Paul Resnick,et al.  The value of reputation on eBay: A controlled experiment , 2002 .