Methodological quality of intervention research in speech–language pathology: Analysis of 10 years of group-design studies

The aim of this work was to critically examine the methodological quality of recent treatment research in speech–language pathology, focusing specifically on group-design studies (randomized, controlled trials and quasi-experimental-design studies). We also considered whether methodological quality differed as a function of a particular study's treatment focus (i.e. literacy, language, speech, fluency, voice), the age group studied (pre-school-age child; school-age child; adult), the year of publication, or the publishing journal. In total, 53 treatment studies published in the last 10 years in three journals of the American Speech–Language–Hearing Association (ASHA) were selected for analysis using Downs and Black's (1998) reliable, valid rating tool for examining methodological quality. The results indicated that the quality of the treatment studies was highly variable in terms of the 25 indicators of quality studied, and that there were few systematic differences in quality attributable to treatment focus, age group studied, year of publication, or the publishing journal. Implications for evidence-based practice and study reporting are discussed.

[1]  C. Bova,et al.  Experimental and quasiexperimental designs , 2002 .

[2]  B. Dodd,et al.  Phonological awareness intervention and the acquisition of literacy skills in children from deprived social backgrounds. , 2005, Language, speech, and hearing services in schools.

[3]  F. Gresham,et al.  Treatment Integrity in Learning Disabilities Intervention Research: Do We Really Know How Treatments Are Implemented? , 2000 .

[4]  Susan B. Neuman,et al.  Single-Subject Experimental Research: Applications for Literacy , 1995 .

[5]  D. Langenberg.  Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction , 2000 .

[6]  N. Black,et al.  The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. , 1998, Journal of epidemiology and community health.

[7]  L. Delbeke.  Quasi-experimentation - design and analysis issues for field settings - cook,td, campbell,dt , 1980 .

[8]  R. Gersten,et al.  Research in Special Education: Scientific Methods and Evidence-Based Practices , 2005 .

[9]  Ralf W. Schlosser,et al.  Evidence-Based Practice in Augmentative and Alternative Communication , 2004 .

[10]  Gary A. Troia,et al.  Phonological Awareness Intervention Research: A Critical Review of the Experimental Methodology , 1999 .

[11]  S. Johnson,et al.  Empirically supported psychosocial interventions for children: an overview. , 1998, Journal of clinical child psychology.

[12]  Specifying interventions in a clinical trial , 2002, The Medical journal of Australia.

[13]  F. Song,et al.  Evaluating non-randomised intervention studies. , 2003, Health technology assessment.

[14]  D. Turpin.  CONSORT and QUOROM guidelines for reporting randomized clinical trials and systematic reviews. , 2005, American journal of orthodontics and dentofacial orthopedics : official publication of the American Association of Orthodontists, its constituent societies, and the American Board of Orthodontics.

[15]  S. Raudenbush.  Learning from Attempts to Improve Schooling: The Contribution of Methodological Diversity , 2005 .

[16]  Christine A. Dollaghan,et al.  The Handbook for Evidence-Based Practice in Communication Disorders , 2007 .

[17]  Y. Lincoln,et al.  Scientific Research in Education , 2004 .

[18]  David Moher,et al.  A checklist to evaluate a report of a nonpharmacological trial (CLEAR NPT) was developed using consensus. , 2005, Journal of clinical epidemiology.

[19]  S. Raudenbush,et al.  Strategies for Improving Precision in Group-Randomized Experiments , 2007 .

[20]  Craig A Champlin,et al.  The efficacy of Fast ForWord Language intervention in school-age children with language impairment: a randomized controlled trial. , 2008, Journal of speech, language, and hearing research : JSLHR.

[21]  Ralf W. Schlosser,et al.  The Efficacy of Augmentative and Alternative Communication: Toward Evidence-Based Practice , 2003 .

[22]  Robert T. Jiménez,et al.  Qualitative Studies in Special Education , 2005 .

[23]  T. Cook,et al.  Quasi-experimentation: Design & analysis issues for field settings , 1979 .

[24]  R. Schlosser.  On the Importance of Being Earnest about Treatment Integrity , 2002 .

[25]  Michael D. Coyne,et al.  Quality Indicators for Group Experimental and Quasi-Experimental Research in Special Education , 2005 .

[26]  William Asher,et al.  Education Psychology, Research Methodology, and Meta-Analysis , 1990 .

[27]  M. Wolery,et al.  The Use of Single-Subject Research to Identify Evidence-Based Practice in Special Education , 2005 .

[28]  R. Newcombe.  Reporting of clinical trials in the JO--the CONSORT Guidelines. , 2000, Journal of orthodontics.

[29]  James Law,et al.  The efficacy of treatment for children with developmental speech and language delay/disorder: a meta-analysis. , 2004, Journal of speech, language, and hearing research : JSLHR.

[30]  B. Tuckman.  Conducting Educational Research , 1972 .