The sorted effects method: discovering heterogeneous effects beyond their averages

The partial (ceteris paribus) e?ects of interest in nonlinear and interactive linear models are heterogeneous as they can vary dramatically with the underlying observed or unobserved covariates. Despite the apparent importance of heterogeneity, a common practice in modern empirical work is to largely ignore it by reporting average partial e?ects (or, at best, average e?ects for some groups, see e.g. Angrist and Pischke (2008)). While average e?ects provide very convenient scalar summaries of typical e?ects, by de?nition they fail to re?ect the entire variety of the heterogenous e?ects. In order to discover these e?ects much more fully, we propose to estimate and report sorted e?ects – a collection of estimated partial e?ects sorted in increasing order and indexed by percentiles. By construction the sorted e?ect curves completely represent and help visualize all of the heterogeneous e?ects in one plot. They are as convenient and easy to report in practice as the conventional average partial e?ects. We also provide a quanti?cation of uncertainty (standard errors and con?dence bands) for the estimated sorted e?ects. We apply the sorted e?ects method to demonstrate several striking patterns of gender-based discrimination in wages, and of race-based discrimination in mortgage lending. Using di?erential geometry and functional delta methods, we establish that the estimated sorted e?ects are consistent for the true sorted e?ects, and derive asymptotic normality and bootstrap approximation results, enabling construction of pointwise con?dence bands (point-wise with respect to percentile indices). We also derive functional central limit theorems and bootstrap approximation results, enabling construction of simultaneous con?dence bands (simultaneous with respect to percentile indices). The derived statistical results in turn rely on establishing Hadamard di?erentiability of the multivariate sorting operator, a result of independent mathematical interest.