Evaluation of Evaluation Studies Using Qualitative Research Methods in the Social Work Literature (1990-2003): Evidence That Constitutes a Wake-Up Call

Objective: This study examines the quality of evaluation studies using qualitative research methods in the social work literature in terms of a number of criteria commonly adopted in the field of qualitative research. Method: Using qualitative and evaluation as search terms, relevant qualitative evaluation studies from 1990 to 2003 indexed by Social Work Abstracts were examined, and their quality was evaluated. Results: The review shows that the quality of published evaluation studies using qualitative research methods in the social work field is not high and that many of the reviewed studies are not sensitive to the following issues: philosophical base of the study, auditability, bias, truth value, consistency, and critical interpretations of the data. Conclusions: Social workers using findings arising from published evaluation studies using qualitative research methods in social work should be cautious and social workers conducting qualitative evaluation studies should be sensitive to the issue of quality. Adequate training for social workers on qualitative evaluation should also be carried out.

[1]  B. Thyer Guidelines for Evaluating Outcome Studies on Social Work Practice , 1991 .

[2]  C. Seale Quality Issues in Qualitative Inquiry , 2002 .

[3]  Christopher J. Brown,et al.  Hermeneutics and the Social Sciences: An Evaluation of the Function of Hermeneutics in a Consumer Disability Study , 1991, The Journal of Sociology & Social Welfare.

[4]  Platt Refocusing children's services: evaluation of an initial assessment process , 2001 .

[5]  J. Fischer Has Mighty Casework Struck Out , 1973 .

[6]  Stephen Andrew Ogden,et al.  Fashionable Nonsense: Postmodern Intellectuals' Abuse of Science , 1999 .

[7]  Timothy B. Kelly,et al.  A Multidimensional Approach to Evaluating School-Linked Services: A School of Social Work and County Public School Partnership , 1998 .

[8]  Karen D. Stout A Continuum of Male Controls and Violence against Women: A Teaching Model , 1991 .

[9]  F. Steier Research and reflexivity , 1991 .

[10]  D. Padgett Qualitative Methods in Social Work Research: Challenges and Rewards , 1998 .

[11]  M. Sandelowski,et al.  The problem of rigor in qualitative research , 1986, ANS. Advances in nursing science.

[12]  Wright,et al.  Family centres: A focus for the children in need debate , 1999 .

[13]  Matthew B. Miles,et al.  Qualitative Data Analysis: An Expanded Sourcebook , 1994 .

[14]  A. Goicoechea-Balbona,et al.  AIDS: the development of a gender appropriate research intervention. , 2000, Social work in health care.

[15]  P. Erera Empathy Training for Helping Professionals , 1997 .

[16]  J. Fischer Is Casework Effective? A Review , 1973 .

[17]  D. Anda A Qualitative Evaluation of a Mentor Program for At-Risk Youth: The Participants' Perspective , 2001 .

[18]  T. C. Walsh Structured process recording: A comprehensive model that incorporates the strengths perspective , 2002 .

[19]  E. Gambrill,et al.  Evidence-Based Practice: An Alternative to Authority-Based Practice * , 1999 .

[20]  T. Gomory Critical Rationalism (Gomory’s Blurry Theory) or Positivism (Thyer’s Theoretical Myopia) , 2001 .

[21]  E. Guba,et al.  Competing paradigms in qualitative research. , 1994 .

[22]  D. Heath Qualitative analysis of private mediation: Benefits for families in public child welfare agencies , 1998 .

[23]  Valerie J. Janesick The dance of qualitative research design: Metaphor, methodolatry, and meaning. , 1994 .

[24]  Pauline Marie Rosenau,et al.  Post-Modernism and the Social Sciences: Insights, Inroads, and Intrusions , 1991 .

[25]  E. Guba,et al.  Paradigmatic Controversies, Contradictions, and Emerging Confluences. , 2005 .

[26]  S. Segal Research on the outcome of social work therapeutic interventions: a review of the literature. , 1972, Journal of health and social behavior.

[27]  David S. Derezotes Evaluation of the Late Nite Basketball Project , 1995 .

[28]  M. Patton,et al.  Qualitative evaluation and research methods , 1992 .

[29]  Thomas A. Rizzo Ethnographic methods and interpretive analysis: Expanding the methodological options of psychologists , 1992 .

[30]  D. Davis,et al.  Ropes course training for youth in a rural setting: “At first I thought it was going to be boring ...” , 1994 .

[31]  A. Rubin Editorial: Social Work Research at the Turn of the Millennium: Progress and Challenges , 2000 .

[32]  T. Gomory A Fallibilistic Response to Thyer’s Theory of Theory-Free Empirical Research in Social Work Practice , 2001 .

[33]  Enola K. Proctor,et al.  Research and Research Training in Social Work: Climate, Connections, and Competencies , 1994 .

[34]  Gore,et al.  A seamless service: meeting the needs of children with disabilities through a multi‐agency approach , 1999 .

[35]  Harland G. Bloland Postmodernism and Higher Education , 1995 .

[36]  Kathleen A. Bolland,et al.  Postmodernism , 2002 .

[37]  Fiona Gardner Critical Reflection in Community-Based Evaluation , 2003 .

[38]  P. Rosenau Post-Modernism and the Social Sciences , 1991 .

[39]  D. Bybee,et al.  The Importance of Context in Understanding Homelessness and Mental Illness: Lessons Learned From a Research Demonstration Project , 1998 .

[40]  M. Hammersley What's Wrong With Ethnography?: Methodological Explorations , 1992 .

[41]  M. Magovčević The Importance of Context , 2003 .

[42]  R. Walsh-Bowers,et al.  Improving Early Adolescents' Peer Relations through Classroom Creative Drama: An Integrated Approach , 1999 .

[43]  Patti Lather Issues of validity in openly ideological research: Between a rock and a soft place , 1986 .

[44]  Jeanne C. Rivard,et al.  The Dynamics of Interagency Collaboration: How Linkages Develop for Child Welfare and Juvenile Justice Sectors in a System of Care Demonstration , 1999, Mental Health Services and Sectors of Care.

[45]  A. Parry Handbook of Qualitative Research , 2002 .

[46]  K. Kirkhart,et al.  Evaluating Effectiveness from the Practitioner Perspective , 1993 .

[47]  Y. Lincoln From Understanding to Action: New Imperatives, New Criteria, New Methods for Interpretive Researchers. , 1998 .

[48]  Robert J. Illback,et al.  Initial Evaluation of a School-Based Integrated Service Program: Kentucky Family Resource and Youth Service Centers. , 1995 .

[49]  E. Whipple,et al.  Doctoral Research Education in a Social Context: Development and Implementation of a Model Outreach Course , 1996 .

[50]  J. Anastas Quality in Qualitative Evaluation: Issues and Possible Answers , 2004 .

[51]  M. Lecompte,et al.  Problems of Reliability and Validity in Ethnographic Research , 1982 .

[52]  Thomas A. Schwandt Farewell to Criteriology , 1996 .

[53]  S. Mathison,et al.  Advantages and Challenges of Using Inclusive Evaluation Approaches in Evaluation Practice , 1998 .

[54]  J. Drisko Strengthening Qualitative Studies and Reports , 1997 .

[55]  B. Thyer First Principles of Practice Research , 1989 .

[56]  Bonnie L. Yegidis,et al.  Comparing Student Evaluations of Distance Learning and Standard Classroom Courses in Graduate Social Work Education , 1999 .

[57]  A. Rubin Practice Effectiveness: More Grounds for Optimism , 1985 .

[58]  W. J. Reid,et al.  Recent evaluations of social work: grounds for optimism , 1982 .

[59]  M. Patton Two Decades of Developments in Qualitative Inquiry , 2002 .