Statistical Challenges in the Evaluation of Treatments for Small Patient Populations

Creative clinical trial designs can help to address the statistical challenges associated with small patient populations. The development of a new treatment typically involves evaluation of its efficacy in a large clinical trial in which patients are randomly assigned either the new treatment or the standard of care. Results from these large randomized clinical trials allow for a definitive and unbiased assessment of the clinical benefit of the new treatment over the standard one. For rare diseases or for small patient subgroups identified within the context of a common disease, it may not be possible to conduct a large randomized trial. In this Review, we discuss alternative clinical study designs and statistical challenges that arise when attempting to assure that study results yield robust conclusions about the safety and effectiveness of a new medical product.

[1]  Jürg Zimmermann,et al.  Effects of a selective inhibitor of the Abl tyrosine kinase on the growth of Bcr–Abl positive cells , 1996, Nature Medicine.

[2]  David L. DeMets,et al.  Data Monitoring Committees in Clinical Trials: A Practical Perspective , 2002 .

[3]  T. Brennan,et al.  The controversy over high-dose chemotherapy with autologous bone marrow transplant for breast cancer. , 2001, Health affairs.

[4]  Hanmin Lee,et al.  A randomized trial of fetal endoscopic tracheal occlusion for severe fetal congenital diaphragmatic hernia. , 2003, The New England journal of medicine.

[5]  Murray K. Clayton,et al.  BAYES DISCRETE SEQUENTIAL BOUNDARIES FOR CLINICAL TRIALS , 2001 .

[6]  Matthias Griese,et al.  A CFTR potentiator in patients with cystic fibrosis and the G551D mutation. , 2011, The New England journal of medicine.

[7]  H. Brachinger,et al.  Decision analysis , 1997 .

[8]  D. DeMets,et al.  Surrogate End Points in Clinical Trials: Are We Being Misled? , 1996, Annals of Internal Medicine.

[9]  A. Griffiths,et al.  Induction and maintenance infliximab therapy for the treatment of moderate-to-severe Crohn's disease in children. , 2007, Gastroenterology.

[10]  P. Armitage Sequential Medical Trials , 1961, Biomedicine / [publiee pour l'A.A.I.C.I.G.].

[11]  J. Abrams,et al.  Much ado about not...enough data: high-dose chemotherapy with autologous stem cell rescue for breast cancer. , 1998, Journal of the National Cancer Institute.

[12]  G. Hortobagyi What is the role of high-dose chemotherapy in the era of targeted therapies? , 2004, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[13]  Daniel,et al.  There are no bad anticancer agents, only bad clinical trial designs--twenty-first Richard and Hinda Rosenthal Foundation Award Lecture. , 1998 .

[14]  J. Blay,et al.  Outcome of patients with advanced gastro-intestinal stromal tumours crossing over to a daily imatinib dose of 800 mg after progression on 400 mg. , 2005, European journal of cancer.

[15]  J. Hampe,et al.  Tumour necrosis factor α and interleukin 1β in relapse of Crohn's disease , 1999, The Lancet.

[16]  A R Feinstein,et al.  The Will Rogers phenomenon. Stage migration and new diagnostic techniques as a source of misleading statistics for survival in cancer. , 1985, The New England journal of medicine.

[17]  G. Ginsburg,et al.  The path to personalized medicine. , 2002, Current opinion in chemical biology.

[18]  Jeffrey W. Clark,et al.  Anaplastic lymphoma kinase inhibition in non-small-cell lung cancer. , 2010, The New England journal of medicine.

[19]  S. Sicherer,et al.  Neonatal-Onset Multisystem Inflammatory Disease Responsive to Interleukin-1β Inhibition , 2007, Pediatrics.

[20]  A. Giobbie-Hurder,et al.  Disease Progression in Hutchinson-Gilford Progeria Syndrome: Impact on Growth and Development , 2007, Pediatrics.

[21]  B. Freidlin,et al.  Outcome--adaptive randomization: is it useful? , 2011, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[22]  J. Kalbfleisch,et al.  The Statistical Analysis of Failure Time Data , 1980 .

[23]  Boris Freidlin,et al.  Overall survival as the outcome for randomized clinical trials with effective subsequent therapies. , 2011, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[24]  Boris Freidlin,et al.  Design issues of randomized phase II trials and a proposal for phase II screening trials. , 2005, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[25]  R. Simon,et al.  Role of independent data-monitoring committees in randomized clinical trials sponsored by the National Cancer Institute. , 1997, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[26]  B. Freidlin,et al.  Monitoring for lack of benefit: a critical component of a randomized clinical trial. , 2009, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[27]  H L Greene,et al.  Mortality and morbidity in patients receiving encainide, flecainide, or placebo. The Cardiac Arrhythmia Suppression Trial. , 1991, The New England journal of medicine.

[28]  D. Alling,et al.  Epsilon aminocaproic acid therapy of hereditary angioneurotic edema. A double-blind study. , 1972, The New England journal of medicine.

[29]  Paul S Albert,et al.  Assessing surrogates as trial endpoints using mixed models , 2005, Statistics in medicine.

[30]  J P Miller,et al.  The relationships among ventricular arrhythmias, left ventricular dysfunction, and mortality in the 2 years after myocardial infarction. , 1984, Circulation.

[31]  N. Barth,et al.  High-dose chemotherapy with autologous stem cell rescue in breast cancer , 2005, Breast Cancer Research and Treatment.

[32]  J. Ruskin,et al.  The cardiac arrhythmia suppression trial (CAST). , 1989, The New England journal of medicine.

[33]  R. Gray,et al.  Multi-Arm Clinical Trials of New Agents: Some Design Considerations , 2008, Clinical Cancer Research.

[34]  Lisa M. McShane,et al.  Effective Incorporation of Biomarkers into Phase II Trials , 2009, Clinical Cancer Research.

[35]  A. Fischer,et al.  Chronic infantile neurological cutaneous and articular syndrome is caused by mutations in CIAS1, a gene highly expressed in polymorphonuclear cells and chondrocytes. , 2002, American journal of human genetics.

[36]  J A Smyth,et al.  Determining optimal therapy--randomized trials in individual patients. , 1986, The New England journal of medicine.

[37]  M J Daniels,et al.  CD4 cell count as a surrogate endpoint in HIV clinical trials: a meta‐analysis of studies of the AIDS Clinical Trials Group , 1998, AIDS.

[38]  M. Baccarani,et al.  Hematologic and cytogenetic responses to imatinib mesylate in chronic myelogenous leukemia. , 2002, The New England journal of medicine.

[39]  R. Demasi,et al.  Meta-Analysis of Antiretroviral Effects on HIV-1 RNA, CD4 Cell Count and Progression to AIDS or Death , 1998, Antiviral therapy.

[40]  T. Johannessen,et al.  Controlled trials in single subjects. 1. Value in clinical medicine. , 1991, BMJ.

[41]  N. Heerema,et al.  Improved early event-free survival with imatinib in Philadelphia chromosome-positive acute lymphoblastic leukemia: a children's oncology group study. , 2009, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[42]  Boris Freidlin,et al.  Conditional power calculations for clinical trials with historical controls , 2006, Statistics in medicine.

[43]  Mark Rothmann,et al.  Approval summary: imatinib mesylate capsules for treatment of adult patients with newly diagnosed philadelphia chromosome-positive chronic myelogenous leukemia in chronic phase. , 2003, Clinical cancer research : an official journal of the American Association for Cancer Research.

[44]  Yoon-Koo Kang,et al.  Trastuzumab in combination with chemotherapy versus chemotherapy alone for treatment of HER2-positive advanced gastric or gastro-oesophageal junction cancer (ToGA): a phase 3, open-label, randomised controlled trial , 2010, The Lancet.

[45]  Francisco Cervantes,et al.  Imatinib compared with interferon and low-dose cytarabine for newly diagnosed chronic-phase chronic myeloid leukemia. , 2003, The New England journal of medicine.

[46]  A. Giobbie-Hurder,et al.  Clinical trial of a farnesyltransferase inhibitor in children with Hutchinson–Gilford progeria syndrome , 2012, Proceedings of the National Academy of Sciences.

[47]  Boris Freidlin,et al.  Reply to Y. Yuan et al , 2011 .

[48]  P. Armitage,et al.  Repeated Significance Tests on Accumulating Data , 1969 .

[49]  Robert H. Bartlett,et al.  Berlin Heart EXCOR Pediatric ventricular assist device Investigational Device Exemption study: study design and rationale. , 2011, American heart journal.

[50]  Boris Freidlin,et al.  Design issues in randomized phase II/III trials. , 2012, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[51]  Rossella Bertulli,et al.  Progression-free survival in gastrointestinal stromal tumours with high-dose imatinib: randomised trial , 2004, The Lancet.

[52]  Eugene H Blackstone,et al.  Prospective trial of a pediatric ventricular assist device. , 2012, The New England journal of medicine.

[53]  S. Senn Controlled trials in single subjects. , 1991, BMJ.

[54]  D. Alling,et al.  Treatment of hereditary angioedema with danazol. Reversal of clinical and biochemical abnormalities. , 1976, The New England journal of medicine.

[55]  R. Gray,et al.  A general inefficacy interim monitoring rule for randomized clinical trials , 2010, Clinical trials.

[56]  Max Halperin,et al.  An Adaptive Procedure for Sequential Clinical Trials , 1969 .

[57]  Boris Freidlin,et al.  Adaptive randomization versus interim monitoring. , 2013, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[58]  B. Freidlin,et al.  Data monitoring committees and interim monitoring guidelines. , 1999, Controlled clinical trials.

[59]  K. K. Lan,et al.  Discrete sequential boundaries for clinical trials , 1983 .

[60]  G. Graff,et al.  Efficacy and tolerability of a new formulation of pancrelipase delayed-release capsules in children aged 7 to 11 years with exocrine pancreatic insufficiency and cystic fibrosis: a multicenter, randomized, double-blind, placebo-controlled, two-period crossover, superiority study. , 2010, Clinical therapeutics.

[61]  H L Greene,et al.  The Cardiac Arrhythmia Suppression Trial: first CAST ... then CAST-II. , 1992, Journal of the American College of Cardiology.

[62]  D. Alling,et al.  Treatment of hereditary angioedema with danazol. Reversal of clinical and biochemical abnormalities. , 1976, The New England journal of medicine.

[63]  D. Sargent,et al.  Survival is not a good outcome for randomized trials with effective subsequent therapies. , 2011, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[64]  G. W. Snedecor STATISTICAL METHODS , 1967 .

[65]  P. O'Brien,et al.  A multiple testing procedure for clinical trials. , 1979, Biometrics.

[66]  S. Holland,et al.  Neonatal-onset multisystem inflammatory disease responsive to interleukin-1beta inhibition. , 2006, The New England journal of medicine.

[67]  S S Ellenberg,et al.  An efficient design for phase III studies of combination chemotherapies. , 1985, Cancer treatment reports.

[68]  Stephen W Lagakos,et al.  Testing and interval estimation for two-sample survival comparisons with small sample sizes and unequal censoring. , 2010, Biostatistics.

[69]  E. Remmers,et al.  De novo CIAS1 mutations, cytokine activation, and evidence for genetic heterogeneity in patients with neonatal-onset multisystem inflammatory disease (NOMID): a new member of the expanding family of pyrin-associated autoinflammatory diseases. , 2002, Arthritis and rheumatism.

[70]  W. Silverman,et al.  The influence of the thermal environment upon the survival of newly born premature infants. , 1958, Pediatrics.

[71]  Controlled trials in single subjects. 2. Limitations of use. , 1991, BMJ.

[72]  E Blacker Some design considerations , 1960 .

[73]  J. Clancy,et al.  Effect of VX-770 in persons with cystic fibrosis and the G551D-CFTR mutation. , 2010, The New England journal of medicine.

[74]  B. Wilcken Rare diseases and the assessment of intervention: What sorts of clinical trials can we use? , 2001, Journal of Inherited Metabolic Disease.

[75]  C. Sawyers,et al.  Efficacy and safety of a specific inhibitor of the BCR-ABL tyrosine kinase in chronic myeloid leukemia. , 2001, The New England journal of medicine.

[76]  E. Korn,et al.  Choice of column scores for testing independence in ordered 2 X K contingency tables. , 1987, Biometrics.

[77]  P. Adamson,et al.  New approaches to drug development in pediatric oncology. , 2005, Cancer journal.

[78]  S. Paul,et al.  Sustained response and prevention of damage progression in patients with neonatal-onset multisystem inflammatory disease treated with anakinra: a cohort study to determine three- and five-year outcomes. , 2012, Arthritis and rheumatism.

[79]  Ying Yuan,et al.  On the usefulness of outcome-adaptive randomization. , 2011, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[80]  Daniel J Sargent,et al.  Disease-free survival versus overall survival as a primary end point for adjuvant colon cancer studies: individual patient data from 20,898 patients on 18 randomized trials. , 2004, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[81]  A. Jaffe,et al.  Risk factors for sudden death after acute myocardial infarction: two-year follow-up. , 1984, The American journal of cardiology.

[82]  J. O'fallon,et al.  Stopping when the experimental regimen does not appear to help. , 1994, Statistics in medicine.

[83]  Laurence L. George,et al.  The Statistical Analysis of Failure Time Data , 2003, Technometrics.

[84]  D. Berry,et al.  Adaptive assignment versus balanced randomization in clinical trials: a decision analysis. , 1995, Statistics in medicine.

[85]  R. D. Boschloo Raised conditional level of significance for the 2 × 2‐table when testing the equality of two probabilities , 1970 .