(Machine) Learning what Policymakers Value ∗

This paper develops a method to uncover the values consistent with observed allocation decisions. We use machine learning estimators for heterogeneous treatment effects to identify who benefits from an allocation. We then decompose the objective underlying the allocation into: differential (i) treatment effects, (ii) welfare weights between entities; and (iii) impact weights across outcomes. We apply this approach to Mexico’s PROGRESA anti-poverty program and estimate the preferences consistent with its design. We find evidence of heterogeneous impacts by income and age; accounting for this heterogeneity, allocations imply higher welfare weights on the indigenous, poor, and for families with more children. The implied value of each missed school day and child sick day is estimated imprecisely but does not rule out conventional valuations or preferences reported by Mexican residents. Alternate eligibility criteria could have improved either average consumption, health or schooling outcomes. ∗We thank Luk Yean and Jolie Wei for excellent research assistance. Thank you to Joseph Cummins, Brian Dillon, John Friedman, Yang Xie, and seminar audiences for helpful conversations. †dan@bjorkegren.com ‡jblumenstock@berkeley.edu §samsun knight@brown.edu

[1]  R. Zeckhauser,et al.  Targeting Transfers through Restrictions on Recipients , 1982 .

[2]  Harry Anthony Patrinos,et al.  Returns to investment in education , 1993 .

[3]  Stephen Morris,et al.  On the Form of Transfers to Special Interests , 1995, Journal of Political Economy.

[4]  B. Davis,et al.  Targeting the Poor in Mexico: An Evaluation of the Selection of Households into PROGRESA , 2001 .

[5]  Petra E. Todd,et al.  International Food Policy Research Institute Randomness in the Experimental Samples of Progresa (education, Health, and Nutrition Program) , 2001 .

[6]  C. Timmins Measuring the dynamic efficiency costs of regulators' preferences: Municipal water utilities in the Arid West , 2002 .

[7]  Paul J. Gertler,et al.  An Experiment in Incentive-Based Welfare: The Impact of PROGRESA on Health in Mexico , 2003 .

[8]  P. Gertler Do Conditional Cash Transfers Improve Child Health? Evidence from PROGRESA’s Control Randomized Experiment. , 2004, The American economic review.

[9]  J. Hoddinott,et al.  The Impact of PROGRESA on Food Consumption , 2004, Economic Development and Cultural Change.

[10]  David J. McKenzie,et al.  Measuring inequality with asset indicators , 2005 .

[11]  D. Coady The Welfare Returns to Finer Targeting: The Case of The Progresa Program in Mexico , 2006 .

[12]  Sergio L. Schmukler,et al.  Emerging Market Instability : Do Sovereign Ratings Affect Country Risk and Stock Returns ? , 1997 .

[13]  E. Skoufias,et al.  Conditional Cash Transfers, Adult Work Incentives, and Poverty , 2006 .

[14]  M. Ravallion How Relevant is Targeting to the Success of an Antipoverty Program? , 2007 .

[15]  Jeffrey A. Smith,et al.  Heterogeneous Impacts in PROGRESA , 2008, SSRN Electronic Journal.

[16]  Edward Miguel,et al.  Spring cleaning: rural water impacts, valuation, and property rights institutions. , 2011, The quarterly journal of economics.

[17]  L. Pritchett,et al.  Estimating Wealth Effects Without Expenditure Data—Or Tears: An Application To Educational Enrollments In States Of India* , 2001, Demography.

[18]  A. Banerjee,et al.  Targeting the Poor: Evidence from a Field Experiment in Indonesia , 2010, The American economic review.

[19]  Toniann Pitassi,et al.  Fairness through awareness , 2011, ITCS '12.

[20]  Nathaniel Hendren Efficient Welfare Weights , 2014 .

[21]  Toru Kitagawa,et al.  Who should be Treated? Empirical Welfare Maximization Methods for Treatment Choice , 2015 .

[22]  Florian Heiss,et al.  Discrete Choice Methods with Simulation , 2016 .

[23]  Stefan Wager,et al.  Policy Learning With Observational Data , 2017, Econometrica.

[24]  Yiling Chen,et al.  Welfare and Distributional Impacts of Fair Classification , 2018, ArXiv.

[25]  Suresh Venkatasubramanian,et al.  Runaway Feedback Loops in Predictive Policing , 2017, FAT.

[26]  Alex Pentland,et al.  Algorithmic Fairness and Efficiency in Targeting Social Welfare Programs at Scale , 2018 .

[27]  Esther Rolf,et al.  Delayed Impact of Fair Machine Learning , 2018, ICML.

[28]  Rema Hanna,et al.  Universal Basic Incomes vs. Targeted Transfers: Anti-Poverty Programs in Developing Countries , 2018, Journal of Economic Perspectives.

[29]  Cyrus Samii,et al.  Evaluating Ex Ante Counterfactual Predictions Using Ex Post Causal Inference , 2018, 1806.07016.

[30]  Stefan Wager,et al.  Estimation and Inference of Heterogeneous Treatment Effects using Random Forests , 2015, Journal of the American Statistical Association.

[31]  François Maniquet,et al.  Optimal Income Taxation Theory and Principles of Fairness , 2018, Journal of Economic Literature.

[32]  Nathan Srebro,et al.  From Fair Decision Making To Social Equality , 2018, FAT.

[33]  S. Greco,et al.  On the Methodological Framework of Composite Indices: A Review of the Issues of Weighting, Aggregation, and Robustness , 2019 .

[34]  Esther Rolf,et al.  Balancing Competing Objectives with Noisy Data: Score-Based Classifiers for Welfare-Aware Machine Learning , 2020, ICML.

[35]  Fan Wang,et al.  The Optimal Allocation of Resources among Heterogeneous Individuals , 2020 .

[36]  M. Biradavolu,et al.  Using Machine Learning and Qualitative Interviews to Design a Five-Question Women's Agency Index , 2021, SSRN Electronic Journal.

[37]  Rediet Abebe,et al.  Fairness, Equality, and Power in Algorithmic Decision-Making , 2021, FAccT.