Insuring the Integrity and Validity of Social Work Interventions: The Case of the Subsidized Guardianship Waiver Experiments

The near loss of the ability to conduct randomized controlled trials in the Title IV-E waiver demonstrations makes the negotiation of a ceasefire urgent in the “causal wars.” Results-oriented accountability is a conceptual framework for managing the micro–macro tensions that arise in social work practice and research. This article uses subsidized guardianship experiments to illustrate the results-oriented accountability process by which innovation is aggregated from the micro level through formative implementation and evaluation into a usable, stable intervention that can be rigorously tested through summative implementation and evaluation at the macro level, and if validated, generalized to the micro level through translative implementation and evaluation.

[1]  R. MacCoun Experimental and Quasi‐Experimental Designs for Generalized Causal Inference, by William R. Shadish, Thomas D. Cook, and Donald T. Campbell. Boston: Houghton Mifflin, 2001, 623 pp., $65.56. , 2003 .

[2]  E. Munro The Impact of Audit on Social Work Practice , 2004 .

[3]  D. Rubin Teaching Statistical Inference for Causal Effects in Experiments and Observational Studies , 2004 .

[4]  Karen A. Blase,et al.  Implementation research: A synthesis of the literature (No. 231). University of South Florida, Louis de la Parte Florida Mental Health Implementation Research Network. , 2012 .

[5]  M. Zelen A new design for randomized clinical trials. , 1979, The New England journal of medicine.

[6]  Karen A. Blase,et al.  Implementation Research: A Synthesis of the , 2005 .

[7]  K. Klein,et al.  The Challenge of Innovation Implementation , 1996 .

[8]  H. Rittel,et al.  Dilemmas in a general theory of planning , 1973 .

[9]  S. Hollon,et al.  Defining empirically supported therapies. , 1998, Journal of consulting and clinical psychology.

[10]  Joshua D. Angrist,et al.  Instrumental variables methods in experimental criminological research: what, why and how , 2005 .

[11]  L. Schorr,et al.  EXPANDING THE EVIDENCE UNIVERSE : Doing Better By Knowing More , 2014 .

[12]  Joshua D. Angrist,et al.  Identification of Causal Effects Using Instrumental Variables , 1993 .

[13]  Eileen Munro,et al.  The limitations of randomized controlled trials in predicting effectiveness. , 2010, Journal of evaluation in clinical practice.

[14]  Gordon H Guyatt,et al.  Clinical expertise in the era of evidence-based medicine and patient choice , 2002, ACP journal club.

[15]  D. Alexander,et al.  The National Institute of Child Health and Human Development , 1965, Nature.

[16]  W. Shadish Philosophy of science and the quantitative-qualitative debates: Thirteen common errors , 1995 .

[17]  Dechang Chen,et al.  The Theory of the Design of Experiments , 2001, Technometrics.

[18]  S. Ellenberg Randomization designs in comparative clinical trials. , 1984, The New England journal of medicine.

[19]  Jo Rycroft-Malone,et al.  An implementation research agenda , 2009, Implementation science : IS.

[20]  Eileen D. Gambrill,et al.  Evidence-Based Practice and Policy: Choices Ahead , 2006 .

[21]  P. Kontos,et al.  Implementation Science BioMed Central Debate , 2009 .

[22]  R. Barth The Move to Evidence-Based Practice: How Well Does it Fit Child Welfare Services? , 2008 .

[23]  P. Holland CAUSAL INFERENCE, PATH ANALYSIS AND RECURSIVE STRUCTURAL EQUATIONS MODELS , 1988 .

[24]  Lawrence A. Palinkas,et al.  Translational Science at the National Institute of Mental Health: Can Social Work Take Its Rightful Place? , 2007 .

[25]  M. Testa,et al.  Permanency Planning Options for Children in Formal Kinship Care. , 1996 .

[26]  M. Scriven A Summative Evaluation of RCT Methodology: & An Alternative Approach to Causal Research , 2008, Journal of MultiDisciplinary Evaluation.

[27]  Iain Chalmers,et al.  Trying to do more Good than Harm in Policy and Practice: The Role of Rigorous, Transparent, Up-to-Date Evaluations , 2003 .

[28]  Christopher Winship,et al.  Counterfactuals and Causal Inference: Methods and Principles for Social Research , 2007 .

[29]  P. Gertler Do Conditional Cash Transfers Improve Child Health? Evidence from PROGRESA’s Control Randomized Experiment. , 2004, The American economic review.

[30]  S. Mccracken,et al.  Practitioner Expertise in Evidence-Based Practice Decision Making , 2008 .

[31]  L. Tickle-Degnen Teaching evidence-based practice. , 2000, The American journal of occupational therapy : official publication of the American Occupational Therapy Association.

[32]  W. Shadish,et al.  Experimental and Quasi-Experimental Designs for Generalized Causal Inference , 2001 .

[33]  D. Sackett,et al.  Evidence based medicine: what it is and what it isn't , 1996, BMJ.

[34]  Howard S. Bloom,et al.  Constructing Instrumental Variables from Experimental Data to Explore How Treatments Produce Effects. , 2005 .

[35]  Giselle Corbie-Smith,et al.  Attitudes and beliefs of african americans toward participation in medical research , 1999, Journal of General Internal Medicine.

[36]  M. Sheppard Practice Validity, Reflexivity and Knowledge for Social Work , 1998 .

[37]  Ruth Gilbert,et al.  Infant sleeping position and the sudden infant death syndrome: systematic review of observational studies and historical review of recommendations from 1940 to 2002. , 2005, International journal of epidemiology.

[38]  M. Scriven Evaluation thesaurus, 4th ed. , 1991 .

[39]  J. Littell How do we know what works? The quality of published reviews of evidence-based practices , 2008 .

[40]  D. Pollio,et al.  Teaching Evidence-Based Practice: Toward a New Paradigm for Social Work Education , 2003 .

[41]  B. McCormack,et al.  What counts as evidence in evidence-based practice? , 2004, Journal of advanced nursing.

[42]  C. Homer Using the Zelen design in randomized controlled trials: debates and controversies. , 2002, Journal of advanced nursing.

[43]  M. Testa Subsidized guardianship: Testing an idea whose time has finally come , 2002 .

[44]  Bethany R. Lee,et al.  Evidence-Based Practice at a Crossroads , 2011 .

[45]  P. Holland Statistics and Causal Inference , 1985 .

[46]  Martin Ravallion,et al.  Should the Randomistas Rule? , 2009 .

[47]  R. Steele,et al.  Practitioner Attitudes Toward Evidence-based Practice: Themes and Challenges , 2006, Administration and Policy in Mental Health and Mental Health Services Research.

[48]  M. Testa,et al.  The gift of kinship foster care , 2002 .

[49]  M. Zelen Strategy and alternate randomized designs in cancer clinical trials. , 1982, Cancer treatment reports.

[50]  Thomas D. Cook,et al.  Why have Educational Evaluators Chosen Not to Do Randomized Experiments? , 2003 .

[51]  Jennifer L. Bellamy,et al.  Teaching Evidence-Based Practice , 2007 .

[52]  Jeffrey M. Woodbridge Econometric Analysis of Cross Section and Panel Data , 2002 .

[53]  P. Holland Causal Inference, Path Analysis and Recursive Structural Equations Models. Program Statistics Research, Technical Report No. 88-81. , 1988 .