(www.interscience.wiley.com) DOI: 10.1002/sim.3337 Variance identification and efficiency analysis in randomized experiments under the matched-pair design

In his 1923 landmark article, Neyman introduced randomization-based inference to estimate average treatment effects from experiments under the completely randomized design. Under this framework, Neyman considered the statistical estimation of the sample average treatment effect and derived the variance of the standard estimator using the treatment assignment mechanism as the sole basis of inference. In this paper, I extend Neyman's analysis to randomized experiments under the matched-pair design where experimental units are paired based on their pre-treatment characteristics and the randomization of treatment is subsequently conducted within each matched pair. I study the variance identification for the standard estimator of average treatment effects and analyze the relative efficiency of the matched-pair design over the completely randomized design. I also show how to empirically evaluate the relative efficiency of the two designs using experimental data obtained under the matched-pair design. My randomization-based analysis differs from previous studies in that it avoids modeling and other assumptions as much as possible. Finally, the analytical results are illustrated with numerical and empirical examples.

[1]  Billewicz Wz,et al.  The efficiency of matched samples: an empirical investigation. , 1965 .

[2]  G. Chase,et al.  On the efficiency of matched pairs in Bernoulli trials , 1968 .

[3]  Rory A. Fisher,et al.  The Arrangement of Field Experiments , 1992 .

[4]  Gary King,et al.  A Politically Robust Experimental Design for Public Policy Evaluation, With Application to the Mexican Universal Health Insurance Program , 2007, Journal of policy analysis and management : [the journal of the Association for Public Policy Analysis and Management].

[5]  Andrew Gelman,et al.  Data Analysis Using Regression and Multilevel/Hierarchical Models , 2006 .

[6]  O S Miettinen,et al.  The matched pairs design in the case of all-or-none responses. , 1968, Biometrics.

[7]  J. Neyman,et al.  Statistical Problems in Agricultural Experimentation , 1935 .

[8]  Jeffrey H Silber,et al.  Optimal multivariate matching before randomization. , 2004, Biostatistics.

[9]  M. B. Wilk,et al.  THE RANDOMZATION ANALYSIS OF A GENERALIZED RANDOMIZED BLOCK DESIGN , 1955 .

[10]  T D Koepsell,et al.  The effect of matching on the power of randomized community intervention studies. , 1993, Statistics in medicine.

[11]  T. Speed,et al.  On the Application of Probability Theory to Agricultural Experiments. Essay on Principles. Section 9 , 1990 .

[12]  Oscar Kempthorne,et al.  THE RANDOMIZATION THEORY OF' EXPERIMENTAL INFERENCE* , 1955 .

[13]  Gary King,et al.  Misunderstandings between experimentalists and observationalists about causal inference , 2008 .

[14]  W G COCHRAN,et al.  Matching in analytical studies. , 1953, American journal of public health and the nation's health.

[15]  E. Stuart,et al.  Misunderstandings among Experimentalists and Observationalists about Causal Inference , 2007 .

[16]  L. H. Youkeles 184 Note: Loss of Power through Ineffective Pairing of Observations in Small Two-Treatment All-or-None Experiments , 1963 .

[17]  D. Rubin [On the Application of Probability Theory to Agricultural Experiments. Essay on Principles. Section 9.] Comment: Neyman (1923) and Causal Inference in Experiments and Observational Studies , 1990 .

[18]  J. I The Design of Experiments , 1936, Nature.

[19]  T. Shakespeare,et al.  Observational Studies , 2003 .

[20]  G. W. Snedecor Statistical Methods , 1964 .

[21]  W. Billewicz The efficiency of matched samples: an empirical investigation. , 1965, Biometrics.

[22]  C. McCulloch,et al.  When does it pay to break the matches for analysis of a matched-pairs design? , 1992, Biometrics.

[23]  Gary King,et al.  The essential role of pair matching in cluster-randomized experiments, with application to the Mexican Universal Health Insurance Evaluation , 2009, 0910.3752.

[24]  P Diehr,et al.  Breaking the matches in a paired t-test for community interventions when the number of pairs is small. , 1995, Statistics in medicine.

[25]  D P Byar,et al.  Assessing the gain in efficiency due to matching in a community intervention study. , 1990, Statistics in medicine.

[26]  P. Holland Statistics and Causal Inference , 1985 .