Evidence-based medicine, heterogeneity of treatment effects, and the trouble with averages.

Evidence-based medicine is the application of scientific evidence to clinical practice. This article discusses the difficulties of applying global evidence ("average effects" measured as population means) to local problems (individual patients or groups who might depart from the population average). It argues that the benefit or harm of most treatments in clinical trials can be misleading and fail to reveal the potentially complex mixture of substantial benefits for some, little benefit for many, and harm for a few. Heterogeneity of treatment effects reflects patient diversity in risk of disease, responsiveness to treatment, vulnerability to adverse effects, and utility for different outcomes. Recognizing these factors, researchers can design studies that better characterize who will benefit from medical treatments, and clinicians and policymakers can make better use of the results.

[1]  Farzad Mostashari,et al.  Epidemic West Nile encephalitis, New York, 1999: results of a household-based seroepidemiological survey , 2001, The Lancet.

[2]  C. Keenan,et al.  Revascularization in coronary artery disease. A review of randomized trial data. , 1998, The Western journal of medicine.

[3]  J. Ioannidis,et al.  Relationship between event rates and treatment effects in clinical site differences within multicenter trials: an example from primary Pneumocystis carinii prophylaxis. , 1999, Controlled clinical trials.

[4]  J. Kahan,et al.  Dissemination of effectiveness and outcomes research. , 1995, Health policy.

[5]  R. D'Agostino Adjustment Methods: Propensity Score Methods for Bias Reduction in the Comparison of a Treatment to a Non‐Randomized Control Group , 2005 .

[6]  A randomized trial of propranolol in patients with acute myocardial infarction. I. Mortality results. , 1982, JAMA.

[7]  D. Sackett,et al.  Evidence based medicine: what it is and what it isn't , 1996, BMJ.

[8]  Jacob Cohen Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[9]  J W Jukema,et al.  The role of a common variant of the cholesteryl ester transfer protein gene in the progression of coronary atherosclerosis. The Regression Growth Evaluation Statin Study Group. , 1998, The New England journal of medicine.

[10]  C. Moore,et al.  Association between presence of HLA-B*5701, HLA-DR7, and HLA-DQ3 and hypersensitivity to HIV-1 reverse-transcriptase inhibitor abacavir , 2002, The Lancet.

[11]  E. M. Hartwell Boston , 1906 .

[12]  S. Pocock,et al.  Subgroup analysis, covariate adjustment and baseline comparisons in clinical trial reporting: current practiceand problems , 2002, Statistics in medicine.

[13]  G. Lowe,et al.  ABC of Atrial Fibrillation: ANTITHROMBOTIC TREATMENT FOR ATRIAL FIBRILLATION , 1996, BMJ.

[14]  L. Walter,et al.  Pitfalls of converting practice guidelines into quality measures: lessons learned from a VA performance measure. , 2004, JAMA.

[15]  Anna C. Balazs,et al.  Influence of adherence to treatment and response of cholesterol on mortality in the coronary drug project. , 1980, The New England journal of medicine.

[16]  Marc Lipsitch,et al.  Geographic diversity and temporal trends of antimicrobial resistance in Streptococcus pneumoniae in the United States , 2003, Nature Medicine.

[17]  A. Roses,et al.  Pharmacogenetics and future drug development and delivery , 2000, The Lancet.

[18]  C. Gross,et al.  Reporting the Recruitment Process in Clinical Trials: Who Are These Patients and How Did They Get There? , 2002, Annals of Internal Medicine.

[19]  A. Hoes,et al.  Penicillin for acute sore throat in children: randomised, double blind trial , 2003, BMJ : British Medical Journal.

[20]  R W Makuch,et al.  Can treatment that is helpful on average be harmful to some patients? A study of the conflicting information needs of clinical inquiry and drug regulation. , 1996, Journal of clinical epidemiology.

[21]  T. Sørensen Which patients may be harmed by good treatments? , 1996, The Lancet.

[22]  R. D'Agostino Propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group. , 2005, Statistics in medicine.

[23]  T. Fahey Applying the results of clinical trials to patients to general practice: perceived problems, strengths, assumptions, and challenges for the future. , 1998, The British journal of general practice : the journal of the Royal College of General Practitioners.

[24]  A. Sinha,et al.  Gene expression profile analysis by DNA microarrays: promise and pitfalls. , 2001, JAMA.

[25]  Daniel R. O’Leary,et al.  The outbreak of West Nile virus infection in the New York City area in 1999. , 2001, The New England journal of medicine.

[26]  Sankey V. Williams,et al.  Hospital and Patient Characteristics Associated With Death After Surgery: A Study of Adverse Occurrence and Failure to Rescue , 1992, Medical care.

[27]  J Lomas,et al.  Words without action? The production, dissemination, and impact of consensus recommendations. , 1991, Annual review of public health.

[28]  W. Rosser,et al.  Application of evidence from randomised controlled trials to general practice , 1999, The Lancet.

[29]  N. Longford Selection bias and treatment heterogeneity in clinical trials. , 1999, Statistics in medicine.

[30]  Nigel T James Scientific method and raw data should be considered , 1996, BMJ.

[31]  P. Rothwell,et al.  Prediction of benefit from carotid endarterectomy in individual patients: a risk-modelling study. European Carotid Surgery Trialists' Collaborative Group. , 1999, Lancet.

[32]  S Senn,et al.  On wisdom after the event. , 1997, Journal of clinical epidemiology.

[33]  Lynn Ordway,et al.  Antecedents of adherence to medical recommendations: Results from the medical outcomes study , 1992, Journal of Behavioral Medicine.

[34]  S. Senn Individual response to treatment: is it a valid assumption? , 2004, BMJ : British Medical Journal.

[35]  R. Weinshilboum Inheritance and drug response. , 2003, The New England journal of medicine.

[36]  D. Rubin,et al.  Reducing Bias in Observational Studies Using Subclassification on the Propensity Score , 1984 .

[37]  K L Baird,et al.  The new NIH and FDA medical research policies: targeting gender, promoting justice. , 1999, Journal of health politics, policy and law.

[38]  A. Laupacis,et al.  Rates of Hyperkalemia after Publication of the Randomized Aldactone Evaluation Study , 2004 .

[39]  Ted J. Kaptchuk,et al.  The Placebo Effect in Alternative Medicine: Can the Performance of a Healing Ritual Have Clinical Significance? , 2002, Annals of Internal Medicine.

[40]  D. Sackett Evidence-based medicine: how to practice and teach EBM: 2nd ed , 2000 .

[41]  J. Lewis,et al.  The case for cross-over trials in phase III. , 1995, Statistics in medicine.

[42]  J M Lachin,et al.  The use of response-adaptive designs in clinical trials. , 1993, Controlled clinical trials.

[43]  An illustrative statistical analysis of cutoff-based randomized clinical trials , 1992 .

[44]  M. Enkin,et al.  Do practice guidelines guide practice? The effect of a consensus statement on the practice of physicians. , 1989, The New England journal of medicine.

[45]  B. Lerer,et al.  Association between the serotonin 2A receptor gene and tardive dyskinesia in chronic schizophrenia , 2001, Molecular Psychiatry.

[46]  P. Little,et al.  Clinical and psychosocial predictors of illness duration from randomised controlled trial of prescribing strategies for sore throat , 1999, BMJ.

[47]  A. Saykin,et al.  The relationship of APOE genotype to neuropsychological performance in long‐term cancer survivors treated with standard dose chemotherapy , 2003, Psycho-oncology.

[48]  S S Ellenberg,et al.  Randomized consent designs for clinical trials: an update. , 1992, Statistics in Medicine.

[49]  B. Pitt,et al.  The effect of spironolactone on morbidity and mortality in patients with severe heart failure. Randomized Aldactone Evaluation Study Investigators. , 1999, The New England journal of medicine.

[50]  T. Shakespeare,et al.  Observational Studies , 2003 .

[51]  J. Wittes,et al.  Analysis and interpretation of treatment effects in subgroups of patients in randomized clinical trials. , 1991, JAMA.

[52]  S. Sindrup,et al.  The pharmacogenetics of codeine hypoalgesia. , 1995, Pharmacogenetics.

[53]  D. Rubin,et al.  The central role of the propensity score in observational studies for causal effects , 1983 .

[54]  B. Pitt,et al.  The Effect of Spironolactone on Morbidity and Mortality in Patients with Severe Heart Failure , 2000 .

[55]  L A Moyé,et al.  Evaluation of ethnic minorities and gender effects in clinical trials: opportunities lost and rediscovered. , 2001, Journal of the National Medical Association.

[56]  G H Guyatt,et al.  N of 1 randomized trials for investigating new drugs. , 1990, Controlled clinical trials.

[57]  G. Taylor,et al.  Determinants of Variable Response to Statin Treatment in Patients With Refractory Familial Hypercholesterolemia , 2001, Arteriosclerosis, thrombosis, and vascular biology.

[58]  P. Lachenbruch Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[59]  M. Mosteller,et al.  Maximizing the value of medicines by including pharmacogenetic research in drug development and surveillance. , 2002, British journal of clinical pharmacology.

[60]  J. A. N. Eedleman,et al.  Nurse-Staffing Levels and the Quality of Care in Hospitals , 2002 .

[61]  T. Vulliamy,et al.  Glucose-6-phosphate dehydrogenase deficiency. , 2000, Bailliere's best practice & research. Clinical haematology.

[62]  C D Naylor,et al.  Subgroups, treatment effects, and baseline risks: some lessons from major cardiovascular trials. , 2000, American heart journal.

[63]  E. Oren,et al.  Potential role of pharmacogenomics in reducing adverse drug reactions: a systematic review. , 2001, JAMA.

[64]  Aniruddha M. Deshpande,et al.  Standardized Reporting of Clinical Practice Guidelines: A Proposal from the Conference on Guideline Standardization , 2003, Annals of Internal Medicine.

[65]  G H Guyatt,et al.  Users' guides to the medical literature: XIV. How to decide on the applicability of clinical trial results to your patient. Evidence-Based Medicine Working Group. , 1998, JAMA.

[66]  S. Satya‐Murti Evidence-based Medicine: How to Practice and Teach EBM , 1997 .

[67]  S. Tanenbaum,et al.  What physicians know. , 1993, The New England journal of medicine.

[68]  A. Berg Chemoprevention of breast cancer: recommendations and rationale. , 2003, The American journal of nursing.

[69]  Siep Thomas,et al.  Attributes of clinical guidelines that influence use of guidelines in general practice: observational study , 1998, BMJ.

[70]  Howard L McLeod,et al.  Pharmacogenomics--drug disposition, drug targets, and side effects. , 2003, The New England journal of medicine.

[71]  G. Lip ABC of atrial fibrillation , 1996 .

[72]  Risk factors for stroke and efficacy of antithrombotic therapy in atrial fibrillation. Analysis of pooled data from five randomized controlled trials. , 1994, Archives of internal medicine.

[73]  H. Petersen,et al.  Combined single subject trials. , 1991, Scandinavian journal of primary health care.

[74]  P. Allmark,et al.  Should research samples reflect the diversity of the population? , 2004, Journal of Medical Ethics.

[75]  G H Guyatt,et al.  Users' guides to the medical literature: XX. Integrating research evidence with the care of the individual patient. Evidence-Based Medicine Working Group. , 2000, JAMA.

[76]  S. Rössner,et al.  Coronary Drug Project Research Group. , 1978, Atherosclerosis.

[77]  A. Hall,et al.  The effect of thiopurine methyltransferase expression on sensitivity to thiopurine drugs. , 2002, Molecular pharmacology.

[78]  N T Longford,et al.  Statistics versus statistical science in the regulatory process. , 1999, Statistics in medicine.

[79]  A. Feinstein,et al.  Problems in the "evidence" of "evidence-based medicine". , 1997, The American journal of medicine.

[80]  Les M Irwig,et al.  An evidence based approach to individualising treatment , 1995, BMJ.

[81]  M Schumacher,et al.  Randomized and non-randomized patients in clinical trials: experiences with comprehensive cohort studies. , 1995, Statistics in medicine.