Campbell and Rubin: A primer and comparison of their approaches to causal inference in field settings.

This article compares Donald Campbell's and Donald Rubin's work on causal inference in field settings on issues of epistemology, theories of cause and effect, methodology, statistics, generalization, and terminology. The two approaches are quite different but compatible, differing mostly in matters of bandwidth versus fidelity. Campbell's work demonstrates broad narrative scope that covers a wide array of concepts related to causation, with a powerful appreciation for human fallibility in making causal judgments, with a more elaborate theory of cause and generalization, and with a preference for design over analysis. Rubin's approach is a more narrow and formal quantitative analysis of effect estimation, sharing a preference for design but best known for analysis, with compelling quantitative approaches to obtaining unbiased quantitative effect estimates from nonrandomized designs and with comparatively little to say about generalization. Much could be gained by joining the emphasis on design in Campbell with the emphasis on analysis in Rubin. However, the 2 approaches also speak modestly different languages that leave some questions about their total commensurability that only continued dialogue can fully clarify.

[1]  N. Léchopier " Experimental and quasi-experimental designs for research on teaching ", de Donald T. Campbell & Julian C. Stanley, (1963). , 2011 .

[2]  J. Bennett,et al.  Enquiry Concerning Human Understanding , 2010 .

[3]  W. Shadish,et al.  The renaissance of field experimentation in evaluating interventions. , 2009, Annual review of psychology.

[4]  Peter M. Steiner,et al.  Can Nonrandomized Experiments Yield Accurate Answers? A Randomized Experiment Comparing Random and Nonrandom Assignments , 2008 .

[5]  Vivian C. Wong,et al.  Three conditions under which experiments and observational studies produce comparable causal estimates: New findings from within‐study comparisons , 2008 .

[6]  Angie Wade Matched Sampling for Causal Effects , 2008 .

[7]  M. Sobel Identification of Causal Parameters in Randomized Studies With Mediating Variables , 2008 .

[8]  Arthur S. Goldberger,et al.  Selection bias in evaluating treatment effects: Some formal illustrations , 2008 .

[9]  Thomas D. Cook,et al.  "Waiting for Life to Arrive": A history of the regression-discontinuity design in Psychology, Statistics and Economics , 2008 .

[10]  Stephen G West,et al.  Doctoral training in statistics, measurement, and methodology in psychology: replication and extension of Aiken, West, Sechrest, and Reno's (1990) survey of PhD programs in North America. , 2008, The American psychologist.

[11]  Steffi Pohl,et al.  Modelling method effects as individual causal effects , 2007 .

[12]  Paul R Rosenbaum,et al.  Combining propensity score matching and group-based trajectory analysis in an observational study. , 2007, Psychological methods.

[13]  Patrick E. McKnight Missing Data: A Gentle Introduction , 2007 .

[14]  David Rindskopf,et al.  Methods for evidence‐based practice: Quantitative synthesis of single‐subject designs , 2007 .

[15]  Keith A. Markus,et al.  Making Things Happen: A Theory of Causal Explanation , 2007 .

[16]  Anne Katz Rn,et al.  A New Perspective , 2003 .

[17]  Charles S Reichardt,et al.  The principle of parallelism in the design of studies to estimate treatment effects. , 2006, Psychological methods.

[18]  J. Robins,et al.  Results of multivariable logistic regression, propensity matching, propensity adjustment, and propensity-based weighting under conditions of nonuniform effect. , 2006, American journal of epidemiology.

[19]  Patrick E. McKnight,et al.  Strengthening research methodology: Psychological measurement and evaluation. , 2006 .

[20]  Stephen W. Raudenbush,et al.  Effects of Kindergarten Retention Policy on Children’s Cognitive Growth in Reading and Mathematics , 2005 .

[21]  William R. Shadish,et al.  Increasing the Degrees of Freedom in Existing Group Randomized Trials , 2005, Evaluation review.

[22]  D. Rubin Causal Inference Using Potential Outcomes , 2005 .

[23]  R. Fisher Statistical methods for research workers , 1927, Protoplasma.

[24]  B. Reskin Including Mechanisms in Our Models of Ascriptive Inequality , 2003, American Sociological Review.

[25]  R. Steyer Analyzing Individual and Average Causal Effects via Structural Equation Models , 2005 .

[26]  D. Rubin Teaching Statistical Inference for Causal Effects in Experiments and Observational Studies , 2004 .

[27]  J. M. Oakes,et al.  The (mis)estimation of neighborhood effects: causal inference for a practicable social epidemiology. , 2004, Social science & medicine.

[28]  Donald B. Rubin,et al.  A Potential Outcomes View of Value-Added Assessment in Education , 2004 .

[29]  Christina A. Christie,et al.  Evaluation Roots: Tracing Theorists′ Views and Influences , 2004 .

[30]  D. Rubin Using Propensity Scores to Help Design Observational Studies: Application to the Tobacco Litigation , 2001, Health Services and Outcomes Research Methodology.

[31]  Donald T. Campbell,et al.  The causal assumptions of quasi-experimental practice , 1986, Synthese.

[32]  Thomas D. Cook,et al.  Causal generalization: How Campbell and Cronbach influenced my theoretical thinking on this topic, including in Shadish, Cook, and Campbell , 2004 .

[33]  Russell V. Lenth,et al.  Statistical Analysis With Missing Data (2nd ed.) (Book) , 2004 .

[34]  Ned Hall,et al.  Causation and counterfactuals , 2004 .

[35]  Steven Glazerman,et al.  Nonexperimental Versus Experimental Estimates of Earnings Impacts , 2003 .

[36]  Melvin M. Mark,et al.  Beyond Use: Understanding Evaluation’s Influence on Attitudes and Actions , 2003 .

[37]  R. Berk Regression Analysis: A Constructive Critique , 2003 .

[38]  W. Shadish,et al.  Content and context: The impact of Campbell and Stanley. , 2003 .

[39]  R. Sternberg The anatomy of impact : what makes the great works of psychology great , 2003 .

[40]  D. Rubin,et al.  Permutation tests for detecting and estimating mixtures in task performance within groups , 2002, Statistics in medicine.

[41]  Xiao-Hua Zhou,et al.  Clustered encouragement designs with individual noncompliance: bayesian inference with randomization, and application to advance directive forms. , 2002, Biostatistics.

[42]  D. Rubin,et al.  Principal Stratification in Causal Inference , 2002, Biometrics.

[43]  F. Mosteller,et al.  Evidence matters : randomized trials in education research , 2002 .

[44]  Ana Ivelisse Avilés,et al.  Linear Mixed Models for Longitudinal Data , 2001, Technometrics.

[45]  J. Hahn,et al.  IDENTIFICATION AND ESTIMATION OF TREATMENT EFFECTS WITH A REGRESSION-DISCONTINUITY DESIGN , 2001 .

[46]  W. Shadish,et al.  Experimental and Quasi-Experimental Designs for Generalized Causal Inference , 2001 .

[47]  D. Rubin,et al.  Contrasts and Correlations in Effect-Size Estimation , 2000, Psychological science.

[48]  P. White Causal attribution and Mill's methods of experimental inquiry: past, present and prospect. , 2000, The British journal of social psychology.

[49]  W. Shadish,et al.  The effects of psychological therapies under clinically representative conditions: a meta-analysis. , 2000, Psychological bulletin.

[50]  A. Dawid,et al.  Causal Inference without Counterfactuals , 2000 .

[51]  Stephen G. West,et al.  Causal inference and generalization in field settings: Experimental and quasi-experimental designs. , 2000 .

[52]  J. Pearl Causality: Models, Reasoning and Inference , 2000 .

[53]  R J Little,et al.  Causal effects in clinical and epidemiological studies via potential outcomes: concepts and analytical approaches. , 2000, Annual review of public health.

[54]  Christopher Winship,et al.  THE ESTIMATION OF CAUSAL EFFECTS FROM OBSERVATIONAL DATA , 1999 .

[55]  P. Rowe What is all the hullabaloo about endostatin? , 1999, The Lancet.

[56]  Charles S. Reichardt,et al.  CHAPTER 5 A Typology of Strategies for Ruling Out Threats to Validity , 1999 .

[57]  W. Shadish,et al.  Design rules: More steps towards a complete theory of quasi-experimentation , 1999 .

[58]  J. Duckart An Evaluation of the Baltimore Community Lead Education and Reduction Corps (CLEARCorps) Program , 1998, Evaluation review.

[59]  J F Troendle Testing for treatment differences with dropouts present in clinical trials--a composite approach. , 1998, Statistics in medicine.

[60]  Xiangen Hu,et al.  A method for exploring the effects of attrition in randomized experiments with dichotomous outcomes , 1998 .

[61]  Stephen B. Jarrell,et al.  Gender Wage Discrimination Bias? A Meta-Regression Analysis , 1998 .

[62]  T. Shakespeare,et al.  Observational Studies , 2003 .

[63]  J. Angrist,et al.  Using Maimonides&Apos; Rule to Estimate the Effect of Class Size on Student Achievement , 1997 .

[64]  W. Vanhonacker Meta-Analysis and Response Surface Extrapolation: A Least Squares Approach , 1996 .

[65]  J. Folkman,et al.  Fighting cancer by attacking its blood supply. , 1996, Scientific American.

[66]  William R. Shadish,et al.  The Social Psychology of Science , 1995 .

[67]  Bruce D. Meyer Natural and Quasi- Experiments in Economics , 1994 .

[68]  Donald T. Campbell,et al.  Retrospective and Prospective on Program Impact Assessment , 1994 .

[69]  K. Delucchi Methods for the analysis of binary outcome results in the presence of missing data. , 1994, Journal of consulting and clinical psychology.

[70]  Joshua D. Angrist,et al.  Identification of Causal Effects Using Instrumental Variables , 1993 .

[71]  Michael W. Neustrom,et al.  THE IMPACT OF DRUNK DRIVING LEGISLATION IN LOUISIANA , 1993 .

[72]  D. Rubin Meta-Analysis: Literature Synthesis or Effect-Size Surface Estimation? , 1992 .

[73]  W. Trochim,et al.  Cutoff assignment strategies for enhancing randomized clinical trials. , 1992, Controlled clinical trials.

[74]  Zita M. Cantwell,et al.  Research methodology: Strengthening causal interpretations of nonexperimental data: L. Sechrest, E. Perrin, and J. Bunker (Eds.). (AHCPR Conference Proceedings, Tucson, AZ, April, 1987). Washington, DC: U.S. Department of Health and Human Services, Public Health Service, Agency for Health Care Polic , 1992 .

[75]  Milbrey W. McLaughlin,et al.  Evaluation and education : at quarter century , 1992 .

[76]  Rory A. Fisher,et al.  The Arrangement of Field Experiments , 1992 .

[77]  P. Rosenbaum Discussing hidden bias in observational studies. , 1991, Annals of internal medicine.

[78]  T. Cook Chapter V: Clarifying the Warrant for Generalized Causal Inferences in Quasi-Experimentation , 1991, Teachers College Record: The Voice of Scholarship in Education.

[79]  Kenneth W. Wachter,et al.  The Future of Meta-Analysis , 1991 .

[80]  P. Rosenbaum,et al.  Sensitivity analysis for matched case-control studies. , 1991, Biometrics.

[81]  W. Shadish,et al.  Foundations of Program Evaluation: Theories of Practice , 1990 .

[82]  T. Speed,et al.  On the Application of Probability Theory to Agricultural Experiments. Essay on Principles. Section 9 , 1990 .

[83]  John B. Willett,et al.  By Design: Planning Research on Higher Education , 1990 .

[84]  Thomas D. Cook,et al.  The generalization of causal connections: Multiple theories in search of clear practice , 1990 .

[85]  Mark W. Lipsey,et al.  Design Sensitivity: Statistical Power for Experimental Research. , 1989 .

[86]  D. Campbell,et al.  Evolving Methods for Enhancing Validity@@@Methodology and Epistemology for Social Science: Selected Papers , 1990 .

[87]  Gary James Jason,et al.  The Logic of Scientific Discovery , 1988 .

[88]  Sanford L. Braver,et al.  Statistical Treatment of the Solomon Four-Group Design: A Meta-Analytic Approach , 1988 .

[89]  P. Holland CAUSAL INFERENCE, PATH ANALYSIS AND RECURSIVE STRUCTURAL EQUATIONS MODELS , 1988 .

[90]  Robert Fildes,et al.  Journal of business and economic statistics 5: Garcia-Ferrer, A. et al., Macroeconomic forecasting using pooled international data, (1987), 53-67 , 1988 .

[91]  H. Farmer A new perspective. , 1988, The Journal of the Florida Medical Association.

[92]  D. Rubin,et al.  Statistical Analysis with Missing Data. , 1989 .

[93]  Stephen G. West,et al.  A Multiplist Strategy for Strengthening Nonequivalent Control Group Designs , 1987 .

[94]  Charles S. Reichardt,et al.  Taking uncertainty into account when estimating effects , 1987 .

[95]  D. Rubin,et al.  Causal Inference in Retrospective Studies , 1987 .

[96]  Martin Bulmer,et al.  Social Science and Social Policy , 2021 .

[97]  W. Trochim,et al.  Advances in Quasi-Experimental Design and Analysis , 1986 .

[98]  David S. Cordray,et al.  Quasi‐experimental analysis: A mixture of methods and judgment , 1986 .

[99]  Donald T. Campbell,et al.  Relabeling Internal and External Validity for Applied Social Scientists. , 1986 .

[100]  Charles S. Reichardt,et al.  Satisfying the constraints of causal modeling , 1986 .

[101]  M. Wittrock Handbook of research on teaching , 1986 .

[102]  P. Holland Statistics and Causal Inference , 1985 .

[103]  R. L. Shotland,et al.  Social Science and Social Policy , 1985 .

[104]  D. Campbell Toward an Epistemologically-Relevant Sociology of Science* , 1985 .

[105]  Thomas D. Cook,et al.  Post-positivist critical multiplism , 1985 .

[106]  Ralph L. Rosnow,et al.  Essentials of Behavioral Research: Methods and Data Analysis , 1984 .

[107]  L. Cronbach,et al.  Designing evaluations of educational and social programs , 1983 .

[108]  M. Brewer,et al.  Scientific Inquiry and the Social Sciences: A Volume in Honor of Donald T. Campbell. , 1983 .

[109]  G. Ferro-Luzzi On Evolutionary Epistemology , 1982, Current Anthropology.

[110]  T. Cook,et al.  Qualitative and quantitative methods in evaluation research , 1981 .

[111]  D. Rogosa A critique of cross-lagged correlation , 1980 .

[112]  T. Cook,et al.  Quasi-experimentation: Design & analysis issues for field settings , 1979 .

[113]  R. Horwitz The planning of observational studies of human populations , 1979 .

[114]  D. Sackett Bias in analytic research. , 1979, Journal of chronic diseases.

[115]  J. Magidson Toward a Causal Model Approach for Adjusting for Preexisting Differences in the Nonequivalent Control Group Situation , 1977 .

[116]  J. Mackie,et al.  The cement of the universe : a study of causation , 1977 .

[117]  D. Rubin Assignment to Treatment Group on the Basis of a Covariate , 1976 .

[118]  R. Swinburne,et al.  The Philosophy of Karl Popper , 1975 .

[119]  D. Campbell III. “Degrees of Freedom” and the Case Study , 1975 .

[120]  D. Rubin Estimating causal effects of treatments in randomized and nonrandomized studies. , 1974 .

[121]  HERMAN T. EPSTEIN,et al.  An Experiment in Education , 1972, Nature.

[122]  H. B. Pepinsky Disadvantaged Child: Compensatory Education, a National Debate. , 1971 .

[123]  Leroy Wolins,et al.  A Procedure for Estimation of Trait, Method, and Error Variance Attributable to a Measure1 2 , 1970 .

[124]  Jerome Hellmuth,et al.  Compensatory education, a national debate , 1970 .

[125]  A. Cornelius Benjamin,et al.  Science, Technology, and Human Values , 1966 .

[126]  T. Kuhn,et al.  The Structure of Scientific Revolutions. , 1964 .

[127]  D. Campbell,et al.  Regression-Discontinuity Analysis: An Alternative to the Ex-Post Facto Experiment , 1960 .

[128]  D. Campbell,et al.  Convergent and discriminant validation by the multitrait-multimethod matrix. , 1959, Psychological bulletin.

[129]  F. Heider The psychology of interpersonal relations , 1958 .

[130]  D. Campbell Factors relevant to the validity of experiments in social settings. , 1957, Psychological bulletin.

[131]  R. Solomon,et al.  An extension of control group design. , 1949, Psychological bulletin.

[132]  Oscar Kempthorne,et al.  Experimental Designs in Sociological Research. , 1949 .

[133]  M. Kendall,et al.  The Logic of Scientific Discovery. , 1959 .

[134]  The Experimental Approach I. The Advantages of Experimental Sociology in the Study of Family Group Patterns , 1932 .