Assessing and reporting heterogeneity in treatment effects in clinical trials: a proposal

Mounting evidence suggests that there is frequently considerable variation in the risk of the outcome of interest in clinical trial populations. These differences in risk will often cause clinically important heterogeneity in treatment effects (HTE) across the trial population, such that the balance between treatment risks and benefits may differ substantially between large identifiable patient subgroups; the "average" benefit observed in the summary result may even be non-representative of the treatment effect for a typical patient in the trial. Conventional subgroup analyses, which examine whether specific patient characteristics modify the effects of treatment, are usually unable to detect even large variations in treatment benefit (and harm) across risk groups because they do not account for the fact that patients have multiple characteristics simultaneously that affect the likelihood of treatment benefit. Based upon recent evidence on optimal statistical approaches to assessing HTE, we propose a framework that prioritizes the analysis and reporting of multivariate risk-based HTE and suggests that other subgroup analyses should be explicitly labeled either as primary subgroup analyses (well-motivated by prior evidence and intended to produce clinically actionable results) or secondary (exploratory) subgroup analyses (performed to inform future research). A standardized and transparent approach to HTE assessment and reporting could substantially improve clinical trial utility and interpretability.

[1]  G. Karthikeyan Clopidogrel and metoprolol in myocardial infarction , 2006, The Lancet.

[2]  John P A Ioannidis,et al.  What makes a good predictor?: the evidence applied to coronary artery calcium score. , 2010, JAMA.

[3]  J. Gold,et al.  Validation of a combined comorbidity index. , 1994, Journal of clinical epidemiology.

[4]  R. Collins,et al.  Addition of clopidogrel to aspirin in 45 852 patients with acute myocardial infarction: randomised placebo-controlled trial , 2005, The Lancet.

[5]  G H Guyatt,et al.  A Consumer's Guide to Subgroup Analyses , 1992, Annals of Internal Medicine.

[6]  C. Furberg,et al.  What do subgroup analyses reveal about differential response to beta-blocker therapy? The Beta-Blocker Heart Attack Trial experience. , 1983, Circulation.

[7]  Ellen Frank,et al.  Moderators of treatment outcomes: clinical, research, and policy importance. , 2006, JAMA.

[8]  J. Ioannidis,et al.  Predictive modeling and heterogeneity of baseline risk in meta-analysis of individual patient data. , 2001, Journal of clinical epidemiology.

[9]  J. Gurwitz,et al.  Risk for Intracranial Hemorrhage after Tissue Plasminogen Activator Treatment for Acute Myocardial Infarction , 1998, Annals of Internal Medicine.

[10]  F E Harrell,et al.  Selection of thrombolytic therapy for individual patients: development of a clinical model. GUSTO-I Investigators. , 1997, American heart journal.

[11]  S. Gutnikov,et al.  From subgroups to individuals: general principles and the example of carotid endarterectomy , 2005, The Lancet.

[12]  E. Braunwald,et al.  Comparison of early invasive and conservative strategies in patients with unstable coronary syndromes treated with the glycoprotein IIb/IIIa inhibitor tirofiban. , 2001, The New England journal of medicine.

[13]  Mark S Roberts,et al.  A framework for tailoring clinical guidelines to comorbidity at the point of care. , 2007, Archives of internal medicine.

[14]  J. Lau,et al.  The impact of high-risk patients on the results of clinical trials. , 1997, Journal of clinical epidemiology.

[15]  M A Waclawiw,et al.  Practical guidelines for multiplicity adjustment in clinical trials. , 2000, Controlled clinical trials.

[16]  J. Ioannidis,et al.  Heterogeneity of the baseline risk within patient populations of clinical trials: a proposed evaluation algorithm. , 1998, American journal of epidemiology.

[17]  S. Yusuf,et al.  Early versus delayed invasive intervention in acute coronary syndromes. , 2009, The New England journal of medicine.

[18]  P. Macfarlane,et al.  West of Scotland Coronary Prevention Study: Identification of high-risk groups and comparison with other cardiovascular intervention trials , 1996 .

[19]  D A Follmann,et al.  A Multivariate Test of Interaction for Use in Clinical Trials , 1999, Biometrics.

[20]  E. Antman,et al.  The TIMI risk score for unstable angina/non-ST elevation MI: A method for prognostication and therapeutic decision making. , 2000, JAMA.

[21]  S. Kaplan,et al.  Comorbidity Affects the Relationship Between Glycemic Control and Cardiovascular Outcomes in Diabetes , 2009, Annals of Internal Medicine.

[22]  J. Habbema,et al.  Subgroup analyses in therapeutic cardiovascular clinical trials: are most of them misleading? , 2006, American heart journal.

[23]  R. Califf,et al.  An independently derived and validated predictive model for selecting patients with myocardial infarction who are likely to benefit from tissue plasminogen activator compared with streptokinase. , 2002, The American journal of medicine.

[24]  S. Pocock,et al.  Subgroup analysis, covariate adjustment and baseline comparisons in clinical trial reporting: current practiceand problems , 2002, Statistics in medicine.

[25]  A. Feinstein,et al.  Problems in the "evidence" of "evidence-based medicine". , 1997, The American journal of medicine.

[26]  David M Kent,et al.  Multivariable risk prediction can greatly enhance the statistical power of clinical trial subgroup analysis , 2006, BMC medical research methodology.

[27]  J. Ioannidis Why Most Published Research Findings Are False , 2005, PLoS medicine.

[28]  M. Rich,et al.  Validation of clinical classification schemes for predicting stroke: results from the National Registry of Atrial Fibrillation. , 2001, JAMA.

[29]  Stephen W Lagakos,et al.  Statistics in medicine--reporting of subgroup analyses in clinical trials. , 2007, The New England journal of medicine.

[30]  P. Rothwell,et al.  Prediction of benefit from carotid endarterectomy in individual patients: a risk-modelling study. European Carotid Surgery Trialists' Collaborative Group. , 1999, Lancet.

[31]  R. Kravitz,et al.  Heterogeneity of treatment effects: implications for guidelines, payment, and quality assessment. , 2007, The American journal of medicine.

[32]  Les M Irwig,et al.  An evidence based approach to individualising treatment , 1995, BMJ.

[33]  S. Yusuf,et al.  Routine vs selective invasive strategies in patients with acute coronary syndromes: a collaborative meta-analysis of randomized trials. , 2005, JAMA.

[34]  D. Kent,et al.  A percutaneous coronary intervention-thrombolytic predictive instrument to assist choosing between immediate thrombolytic therapy versus delayed primary percutaneous coronary intervention for acute myocardial infarction. , 2008, The American journal of cardiology.

[35]  D. Kent,et al.  Comparison of mortality benefit of immediate thrombolytic therapy versus delayed primary angioplasty for acute myocardial infarction. , 2007, The American journal of cardiology.

[36]  D. Kent,et al.  Progression risk, urinary protein excretion, and treatment effects of angiotensin-converting enzyme inhibitors in nondiabetic kidney disease. , 2007, Journal of the American Society of Nephrology : JASN.

[37]  Frank Davidoff,et al.  Heterogeneity is not always noise: lessons from improvement. , 2009, JAMA.

[38]  C. Warlow,et al.  Prediction of benefit from carotid endar terectomy in individual patients: a risk-modelling study , 1999, The Lancet.

[39]  D. Bild,et al.  Score What Makes a Good Predictor ? : The Evidence Applied to Coronary Artery Calcium , 2010 .

[40]  Sara T Brookes,et al.  Subgroup analyses in randomized trials: risks of subgroup-specific analyses; power and sample size for the interaction test. , 2004, Journal of clinical epidemiology.

[41]  D. Kent,et al.  Are randomized controlled trials sufficient evidence to guide clinical practice in Type II (non-insulin-dependent) diabetes mellitus? , 2000, Diabetologia.

[42]  C. Vassanelli,et al.  [Comparison of early invasive and conservative strategies in patients with unstable coronary syndromes treated with the glycoprotein IIb/IIIa inhibitor tirofiban]. , 2001, Italian heart journal. Supplement : official journal of the Italian Federation of Cardiology.

[43]  J. Wittes,et al.  Analysis and interpretation of treatment effects in subgroups of patients in randomized clinical trials. , 1991, JAMA.

[44]  E. Antman,et al.  An integrated clinical approach to predicting the benefit of tirofiban in non-ST elevation acute coronary syndromes. Application of the TIMI Risk Score for UA/NSTEMI in PRISM-PLUS. , 2002, European heart journal.

[45]  L. Caplan,et al.  Evidence based medicine: concerns of a clinical neurologist , 2001, Journal of neurology, neurosurgery, and psychiatry.

[46]  D. Kent,et al.  Are Some Patients Likely to Benefit From Recombinant Tissue-Type Plasminogen Activator for Acute Ischemic Stroke Even Beyond 3 Hours From Symptom Onset? , 2003, Stroke.

[47]  P. Rothwell Subgroup analysis in randomised controlled trials: importance, indications, and interpretation , 2005, The Lancet.

[48]  S. Yusuf,et al.  Telmisartan to prevent recurrent stroke and cardiovascular events. , 2008, The New England journal of medicine.

[49]  Richard L Kravitz,et al.  Dealing with heterogeneity of treatment effects: is the literature up to the challenge? , 2009, Trials.

[50]  B. Gage,et al.  Selecting Patients With Atrial Fibrillation for Anticoagulation: Stroke Risk Stratification in Patients Taking Aspirin , 2004, Circulation.

[51]  G. Smith,et al.  The ‘number need to treat’: does it help clinical decision making? , 1999, Journal of Human Hypertension.

[52]  David M Kent,et al.  Limitations of applying summary results of clinical trials to individual patients: the need for risk stratification. , 2007, JAMA.

[53]  T. Peters,et al.  Subgroup analyses in randomised controlled trials: quantifying the risks of false-positives and false-negatives. , 2001, Health technology assessment.

[54]  C D Naylor,et al.  Subgroups, treatment effects, and baseline risks: some lessons from major cardiovascular trials. , 2000, American heart journal.

[55]  Jeffrey M Albert,et al.  Assessing Treatment Effect Heterogeneity in Clinical Trials with Blocked Binary Outcomes , 2005, Biometrical journal. Biometrische Zeitschrift.

[56]  D. Lubeck,et al.  Assessment of prognosis with the total illness burden index for prostate cancer , 2007, Cancer.

[57]  D. Kent,et al.  Competing risk and heterogeneity of treatment effect in clinical trials , 2008, Trials.

[58]  T. Chalmers,et al.  Effect of coronary artery bypass graft surgery on survival: overview of 10-year results from randomised trials by the Coronary Artery Bypass Graft Surgery Trialists Collaboration , 1994, The Lancet.

[59]  L. Køber,et al.  Simple Risk Stratification at Admission to Identify Patients With Reduced Mortality From Primary Angioplasty , 2005, Circulation.

[60]  Peter Fayers,et al.  Can overall results of clinical trials be applied to all patients? , 1995, The Lancet.

[61]  D. Kent,et al.  Reporting clinical trial results to inform providers, payers, and consumers. , 2005, Health affairs.

[62]  D. Black The Limitations of Evidence , 2015, Journal of the Royal College of Physicians of London.

[63]  A. H. Feiveson,et al.  Power by Simulation , 2002 .

[64]  S. Lange,et al.  Adjusting for multiple testing--when and how? , 2001, Journal of clinical epidemiology.

[65]  Furberg Cd,et al.  What do subgroup analyses reveal about differential response to beta-blocker therapy? The Beta-Blocker Heart Attack Trial experience. , 1983 .

[66]  L. Hillis,et al.  Optimal management of acute coronary syndromes. , 2009, The New England journal of medicine.

[67]  J. Ioannidis,et al.  Assessment of claims of improved prediction beyond the Framingham risk score. , 2009, JAMA.

[68]  Jonathan D Mahnken,et al.  Development of a contemporary bleeding risk model for elderly warfarin recipients. , 2006, Chest.

[69]  Richard L Kravitz,et al.  Evidence-based medicine, heterogeneity of treatment effects, and the trouble with averages. , 2004, The Milbank quarterly.

[70]  S. Assmann,et al.  Subgroup analysis and other (mis)uses of baseline data in clinical trials , 2000, The Lancet.

[71]  S. Pocock,et al.  More on subgroup analyses in clinical trials. , 2008, The New England journal of medicine.

[72]  R. Hayward,et al.  Beyond the Randomized Clinical Trial the Role of Effectiveness Studies in Evaluating Cardiovascular Therapies the Achilles' Heel of Rcts Key Issues in Outcomes Research , 2022 .

[73]  I. Tannock,et al.  False-positive results in clinical trials: multiple significance tests and the problem of unreported comparisons. , 1996, Journal of the National Cancer Institute.

[74]  R. Rossaint,et al.  Drotrecogin alfa (activated) for adults with severe sepsis and a low risk of death. , 2005, The New England journal of medicine.

[75]  Xin Sun,et al.  Is a subgroup effect believable? Updating criteria to evaluate the credibility of subgroup analyses , 2010, BMJ : British Medical Journal.

[76]  T. Lancet,et al.  West of Scotland Coronary Prevention Study: identification of high-risk groups and comparison with other cardiovascular intervention trials , 1996, The Lancet.

[77]  Gordon H Guyatt,et al.  Can we individualize the 'number needed to treat'? An empirical study of summary effect measures in meta-analyses. , 2002, International journal of epidemiology.