Testing a test

In this issue of Neurology , two reports describe the use of MRI in acute stroke.1,2 Both conclude that diffusion-weighted imaging (DWI) is better than CT or conventional MRI for patient care. How should the data be evaluated? If these were treatment trials, the answer would be straightforward. Prospective, randomized, blinded, placebo-controlled trials designed to minimize bias and carried out with a sufficient number of patients and an important clinical endpoint are widely accepted as the best evidence for treatment efficacy. Other study designs are more subject to bias and provide weaker evidence. Specific criteria based on assessment of the quality of the published evidence are used to evaluate therapies and provide recommendations for patient care.3-5 Similar evidence-based standards exist for the evaluation of studies of diagnostic tests. Proper study design to minimize bias is as critical for evaluation of diagnostic tests as it is for the evaluation of new treatments. Studies with methodologic shortcomings tend to overestimate the accuracy of diagnostic tests.6 The optimal design for assessing the accuracy of a diagnostic test is a prospective, blinded comparison of the test to a reference standard in a consecutive series of patients from a relevant clinical population.6 Prospective studies with predefined endpoints are necessary to prevent the bias that occurs when results are used to guide the selection of endpoints or establish criteria for a positive test result. Even subsequent blinded reading cannot overcome this problem. If the comparison is to an existing technology, patients must be selected before either test is done; the tests must be performed close together in time, obtained in random order, and interpreted independently of each other.7,8 There are a variety of methodologic aspects to proper study design that are specific to the evaluation …

[1]  G. Guyatt,et al.  Grades of recommendation for antithrombotic agents. , 2001, Chest.

[2]  M. Moseley,et al.  Yield of diffusion-weighted MRI for detection of potentially relevant findings in stroke patients , 2000, Neurology.

[3]  C. Beaulieu,et al.  Comparison of diffusion-weighted MRI and CT in acute stroke , 2000, Neurology.

[4]  R. Holloway,et al.  To test or not to test? , 1999, Neurology.

[5]  P. Bossuyt,et al.  Empirical evidence of design-related bias in studies of diagnostic tests. , 1999, JAMA.

[6]  J M Zabramski,et al.  Guidelines for the management of spontaneous intracerebral hemorrhage: A statement for healthcare professionals from a special writing group of the Stroke Council, American Heart Association. , 1999, Stroke.

[7]  J E Heffner,et al.  Methodologic standards for diagnostic test research in pulmonary medicine. , 1998, Chest.

[8]  M. Kaste,et al.  Intravenous thrombolysis with recombinant tissue plasminogen activator for acute hemispheric stroke. The European Cooperative Acute Stroke Study (ECASS) , 1995, JAMA.

[9]  A R Feinstein,et al.  Use of methodological standards in diagnostic test research. Getting better but still not good. , 1995, JAMA.

[10]  G. Guyatt,et al.  Users' Guides to the Medical Literature: III. How to Use an Article About a Diagnostic Test: B. What Are the Results and Will They Help Me In Caring for My Patients? , 1994 .

[11]  Gordon H. Guyatt,et al.  Users' Guides to the Medical Literature: III. How to Use an Article About a Diagnostic Test A. Are the Results of the Study Valid? , 1994 .

[12]  D L Kent,et al.  Disease, level of impact, and quality of research methods. Three dimensions of clinical efficacy assessment applied to magnetic resonance imaging. , 1992, Investigative radiology.

[13]  L. Hall,et al.  Magnetic resonance imaging of the head and spine: effective for the clinician or the patient? , 1991, BMJ.

[14]  PET scans and technology assessment. , 1988, JAMA.