Toward a Conceptual Framework for Mixed-Method Evaluation Designs

In recent years evaluators of educational and social programs have expanded their methodological repertoire with designs that include the use of both qualitative and quantitative methods. Such practice, however, needs to be grounded in a theory that can meaningfully guide the design and implementation of mixed-method evaluations. In this study, a mixed-method conceptual framework was developed from the theoretical literature and then refined through an analysis of 57 empirical mixed-method evaluations. Five purposes for mixed-method evaluations are identified in this conceptual framework: triangulation, complementarity, development, initiation, and expansion. For each of the five purposes, a recommended design is also presented in terms of seven relevant design characteristics. These design elements encompass issues about methods, the phenomena under investigation, paradigmatic framework, and criteria for implementation. In the empirical review, common misuse of the term triangulation was apparent in evaluations that stated such a purpose but did not employ an appropriate design. In addition, relatively few evaluations in this review integrated the different method types at the level of data analysis. Strategies for integrated data analysis are among the issues identified as priorities for further mixed-method work.

[1]  D. Campbell,et al.  Convergent and discriminant validation by the multitrait-multimethod matrix. , 1959, Psychological bulletin.

[2]  S. Sieber.  The Integration of Fieldwork and Survey Methods , 1973, American Journal of Sociology.

[3]  N. Denzin.  The research act: A theoretical introduction to sociological methods , 1977 .

[4]  M. Trend,et al.  On The Reconciliation of Qualitative and Quantitative Analyses: A Case Study , 1978 .

[5]  T. Cook,et al.  Beyond qualitative versus quantitative methods , 1979 .

[6]  T. Jick.  Mixing Qualitative and Quantitative Methods: Triangulation in Action. , 1979 .

[7]  T. Cook,et al.  Qualitative and quantitative methods in evaluation research , 1981 .

[8]  Evaluation and Comparison: Social Learning Curriculum and Instrumental Enrichment. Final Report. , 1981 .

[9]  Policy Researcher as Sleuth: New Approaches to Integrating Qualitative and Quantitative Methods. , 1981 .

[10]  Doren L. Madey,et al.  Some Benefits of Integrating Qualitative and Quantitative Methods in Program Evaluation, with Illustrations , 1982 .

[11]  Multimethod Policy Research: Issues and Applications. , 1982 .

[12]  D. Fetterman.  Ethnography in Educational Evaluation , 1984 .

[13]  Gretchen B. Rossman,et al.  Combining Quantitative and Qualitative Methods in a Single Large-Scale Evaluation Study , 1985 .

[14]  Thomas D. Cook,et al.  Post-positivist critical multiplism , 1985 .

[15]  Linda S. Lotto.  Qualitative Data Analysis: A Sourcebook of New Methods , 1986 .

[16]  M. Peters,et al.  The Development and Trials of a Decision-Making Model , 1986 .

[17]  Triangulation in the Evaluation of School System Interventions. , 1986 .

[18]  David M. Fetterman,et al.  Educational Evaluation: Ethnography in Theory, Practice, and Politics , 1986 .

[19]  M. Smith,et al.  The Whole is Greater: Combining Qualitative and Quantitative Approaches in Evaluation Studies. , 1986 .

[20]  Martin Bulmer,et al.  Social Science and Social Policy , 2021 .

[21]  John K. Smith,et al.  Closing Down the Conversation: The End of the Quantitative-Qualitative Debate Among Educational Inquirers , 1986 .

[22]  T. Kenneth Moran,et al.  Research and Managerial Strategies for Integrating Evaluation Research Into Agency Decision Making , 1987 .

[23]  Melvin M. Mark,et al.  Alternative models for the use of multiple methods , 1987 .

[24]  Melvin M. Mark,et al.  Improving Inferences from Multiple Methods. , 1987 .

[25]  M. Fine,et al.  Qualitative and quantitative methods: When stories converge , 1987 .

[26]  Charles S. Reichardt,et al.  Taking uncertainty into account when estimating effects , 1987 .

[27]  Sandra Mathison,et al.  Why Triangulate? , 1988 .