A scale of methodological quality for clinical studies of radiologic examinations.

PURPOSE To develop and evaluate a scale for assessment of the methodological quality for clinical investigations of radiologic studies. MATERIALS AND METHODS A scale was developed that included methodological standards compiled from established sources for assessing the methodological quality of study designs in clinical research and characteristics related to biases commonly observed in clinical radiologic research. The scale was composed of 15 standards and was tested with the results of 96 studies on imaging of liver hemangioma. Interrater reliability was measured between two observers by using percentage agreement and kappa statistics. Interrater reliability between two observers for a composite quality index that encompassed the 15 standards was measured with the intraclass correlation coefficient. RESULTS Agreement between the two observers was almost perfect (kappa value, 0.8-1.0) for 11 standards and substantial (kappa value, 0.74-0.78) for four standards. Agreement between the observers with regard to the composite quality index also was high (intraclass correlation coefficient r, 0.91 [95% CI: 0.87, 0.94]). CONCLUSION The scale appears to be reliable for the assessment of methodological quality of clinical investigations of radiologic studies.

[1]  S. Pocock,et al.  Statistical problems in the reporting of clinical trials. A survey of three medical journals. , 1987, The New England journal of medicine.

[2]  A R Feinstein,et al.  Use of methodological standards in diagnostic test research. Getting better but still not good. , 1995, JAMA.

[3]  K A L'Abbé,et al.  Meta-analysis in clinical research. , 1987, Annals of internal medicine.

[4]  G. Guyatt,et al.  Users' guides to the medical literature. III. How to use an article about a diagnostic test. B. What are the results and will they help me in caring for my patients? The Evidence-Based Medicine Working Group. , 1994, JAMA.

[5]  A. Feinstein,et al.  Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. , 1978, The New England journal of medicine.

[6]  S. Siegelman,et al.  Improving radiology research methods: what is being asked and who is being studied? , 1997, Radiology.

[7]  A R Jadad,et al.  Guides for reading and interpreting systematic reviews: II. How did the authors find the studies and assess their quality? , 1998, Archives of pediatrics & adolescent medicine.

[8]  S. Mckenzie.  Reviewing scientific papers. , 1995, Archives of disease in childhood.

[9]  B. McNeil,et al.  Rapid method for rigorous assessment of radiologic imaging technologies. , 1997, Radiology.

[10]  G. Grégoire,et al.  Discrepancies between meta-analyses and subsequent large randomized, controlled trials. , 1997, The New England journal of medicine.

[11]  V. Bramwell,et al.  Do authors of review articles use systematic methods to identify, assess and synthesize information? , 1997, Annals of oncology : official journal of the European Society for Medical Oncology.

[12]  P. Seglen,et al.  Citations and journal impact factors: questionable indicators of research quality , 1997, Allergy.

[13]  Gordon H. Guyatt,et al.  Users' Guides to the Medical Literature: III. How to Use an Article About a Diagnostic Test A. Are the Results of the Study Valid? , 1994 .

[14]  F. Mosteller,et al.  Reporting on methods in clinical trials. , 2019, The New England journal of medicine.

[15]  Frederick Mosteller,et al.  Guidelines for Meta-analyses Evaluating Diagnostic Tests , 1994, Annals of Internal Medicine.

[16]  D. Cook,et al.  Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? , 1998, The Lancet.

[17]  Curtis L. Meinert,et al.  A proposal for structured reporting of randomized controlled trials. The Standards of Reporting Trials Group. , 1994, JAMA.

[18]  R. J. Hayes,et al.  Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. , 1995, JAMA.

[19]  A R Jadad,et al.  Assessing the quality of reports of randomized clinical trials: is blinding necessary? , 1996, Controlled clinical trials.

[20]  T C Chalmers,et al.  A method for assessing the quality of a randomized control trial. , 1981, Controlled clinical trials.

[21]  C. Mulrow.  The medical review article: state of the science. , 1987, Annals of internal medicine.

[22]  D Moher,et al.  CONSORT: an evolving tool to help improve the quality of reports of randomized controlled trials. Consolidated Standards of Reporting Trials. , 1998, JAMA.

[23]  L M Bouter,et al.  Method guidelines for systematic reviews in the Cochrane Collaboration Back Review Group for Spinal Disorders. , 1997, Spine.

[24]  B. McNeil,et al.  Assessment of radiologic tests: control of bias and other design considerations. , 1988, Radiology.

[25]  M McCally,et al.  The poor quality of early evaluations of magnetic resonance imaging. , 1988, JAMA.

[26]  G H Guyatt,et al.  Agreement among reviewers of review articles. , 1991, Journal of clinical epidemiology.

[27]  G. Guyatt,et al.  Users' Guides to the Medical Literature: III. How to Use an Article About a Diagnostic Test: B. What Are the Results and Will They Help Me In Caring for My Patients? , 1994 .

[28]  K. Dickersin,et al.  Systematic Reviews: Identifying relevant studies for systematic reviews , 1994 .

[29]  G. Guyatt,et al.  Validation of an index of the quality of review articles. , 1991, Journal of clinical epidemiology.

[30]  B. Everitt,et al.  Statistical methods for rates and proportions , 1973 .

[31]  D. Haynor,et al.  The Clinical Efficacy of Magnetic Resonance Imaging in Neuroimaging , 1994, Annals of Internal Medicine.

[32]  J. Fleiss,et al.  Intraclass correlations: uses in assessing rater reliability. , 1979, Psychological bulletin.

[33]  K. Seers,et al.  Development of a tool to rate the quality assessment of randomized controlled trials using a Delphi technique. , 1997, Journal of advanced nursing.

[34]  T. Chalmers,et al.  Meta-analyses of randomized controlled trials. , 1987, The New England journal of medicine.

[35]  J R Thornbury,et al.  Eugene W. Caldwell Lecture. Clinical efficacy of diagnostic imaging: love it or leave it. , 1994, AJR. American journal of roentgenology.

[36]  M. Cho,et al.  Instruments for assessing the quality of drug studies published in the medical literature. , 1994, JAMA.

[37]  J. Bailar.  The promise and problems of meta-analysis. , 1997, The New England journal of medicine.

[38]  A. Jadad,et al.  The importance of quality of primary studies in producing unbiased systematic reviews. , 1996, Archives of internal medicine.

[39]  Dennis G. Fryback,et al.  The Efficacy of Diagnostic Imaging , 1991, Medical decision making : an international journal of the Society for Medical Decision Making.

[40]  C. Mulrow,et al.  Systematic Reviews: Rationale for systematic reviews , 1994 .

[41]  A R Jadad,et al.  Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. , 1995, Controlled clinical trials.