Sequential Testing Theory and Stochastic Optimization Over Time

The first seven sections of this chapter give an overview of the theory of fully sequential tests, starting with simple hypotheses involving likelihood ratio statistics and then extending the theory to composite hypotheses via generalized likelihood ratio (GLR) statistics. This theory is of particular relevance to Sect. 5.2 on vaccine clinical trials. The theory is modified in Sect. 3.5 for group sequential designs and later for adaptive designs in Sect. 8.2.3. The classic result of Wald and Wolfowitz on the optimality of the sequential probability ratio test is derived in Sect. 3.6 by using dynamic programming that is also introduced in that section for general stochastic optimization over time. Dynamic programming is often difficult to implement directly for nonlinear models and approximate dynamic programming (ADP) methods have been developed to overcome the computational and analytical difficulties.

[1]  T. Fleming,et al.  Parameter estimation following group sequential hypothesis testing , 1990 .

[2]  T. Lai.  Sequential changepoint detection in quality control and dynamical systems , 1995 .

[3]  Tze Leung Lai,et al.  Asymptotic Expansions for the Distributions of Stopped Random Walks and First Passage Times , 1994 .

[4]  J. Denne,et al.  Sample size recalculation using conditional power , 2001, Statistics in medicine.

[5]  John Whitehead,et al.  Confidence intervals for secondary parameters following a sequential test , 2000 .

[6]  D. Siegmund,et al.  Using the Generalized Likelihood Ratio Statistic for Sequential Detection of a Change-Point , 1995 .

[7]  D. Bates,et al.  Approximations to the Log-Likelihood Function in the Nonlinear Mixed-Effects Model , 1995 .

[8]  J Whitehead,et al.  Group sequential clinical trials with triangular continuation regions. , 1983, Biometrics.

[9]  W. J. Studden,et al.  Theory Of Optimal Experiments , 1972 .

[10]  E. Vonesh,et al.  A note on the use of Laplace's approximation for nonlinear mixed-effects models , 1996 .

[11]  Gernot Wassmer,et al.  A Brief Review on Software Developments for Group Sequential and Adaptive Designs , 2006, Biometrical journal. Biometrische Zeitschrift.

[12]  L V Rubinstein,et al.  A comparison of two phase I trial designs. , 1994, Statistics in medicine.

[13]  R. Wolfinger,et al.  Generalized linear mixed models a pseudo-likelihood approach , 1993 .

[14]  N. Breslow,et al.  Approximate inference in generalized linear mixed models , 1993 .

[15]  K. K. Lan,et al.  Discrete sequential boundaries for clinical trials , 1983 .

[16]  J. Wittes,et al.  The role of internal pilot studies in increasing the efficiency of clinical trials. , 1990, Statistics in medicine.

[17]  T. Lai,et al.  Sequential generalized likelihood ratio tests for vaccine safety evaluation , 2010, Statistics in medicine.

[18]  T. Lai,et al.  Hybrid resampling methods for confidence intervals , 1998 .

[19]  Qing Liu,et al.  A note on Gauss—Hermite quadrature , 1994 .

[20]  E. S. Page.  CONTINUOUS INSPECTION SCHEMES , 1954 .

[21]  D. Siegmund.  Estimation following sequential tests , 1978 .

[22]  K M Facey,et al.  A sequential procedure for a phase II efficacy trial in hypercholesterolemia. , 1992, Controlled clinical trials.

[23]  A. Tsiatis,et al.  Approximately optimal one-parameter boundaries for group sequential trials. , 1987, Biometrics.

[24]  W. A. Wallis.  The Statistical Research Group, 1942–1945 , 1980 .

[25]  E B Wilson,et al.  The Determination of L.D.50 and Its Sampling Error in Bio-Assay. , 1943, Proceedings of the National Academy of Sciences of the United States of America.

[26]  C Jennison,et al.  Estimating the sample size for a t-test using an internal pilot. , 1999, Statistics in medicine.

[27]  Sue-Jane Wang,et al.  Modification of Sample Size in Group Sequential Clinical Trials , 1999, Biometrics.

[28]  J. Whitehead.  Supplementary analysis at the conclusion of a sequential clinical trial. , 1986, Biometrics.

[29]  Steven A. Julious,et al.  Moving statistics beyond the individual clinical trial: applying decision science to optimize a clinical development plan , 2005 .

[30]  B. Efron.  Better Bootstrap Confidence Intervals , 1987 .

[31]  J. Wolfowitz,et al.  Optimum Character of the Sequential Probability Ratio Test , 1948 .

[32]  Tze Leung Lai,et al.  Tests and confidence intervals for secondary endpoints in sequential clinical trials , 2009 .

[33]  Sung C. Choi,et al.  Early decision in clinical trials when the treatment differences are small. Experience of a controlled trial in head trauma. , 1985, Controlled clinical trials.

[34]  M. Kulldorff,et al.  A conditional maximized sequential probability ratio test for Pharmacovigilance , 2010, Statistics in medicine.

[35]  N. Mantel.  Evaluation of survival data and two new rank order statistics arising in its consideration. , 1966, Cancer chemotherapy reports.

[36]  Jay Bartroff,et al.  Incorporating Individual and Collective Ethics into Phase I Cancer Trial Designs , 2011, Biometrics.

[37]  R. Byington,et al.  Beta-blocker heart attack trial: design, methods, and baseline results. Beta-blocker heart attack trial research group. , 1984, Controlled clinical trials.

[38]  Anastasios A. Tsiatis,et al.  The asymptotic joint distribution of the efficient scores test for the proportional hazards model calculated over time , 1981 .

[39]  Robert T. Chen.  Special Methodological Issues in Pharmacoepidemiology Studies of Vaccine Safety , 2002 .

[40]  John O'Quigley,et al.  Continual reassessment designs with early termination. , 2002, Biostatistics.

[41]  N. Schmitz.  Optimal Sequentially Planned Decision Procedures , 1992 .

[42]  Tze Leung Lai,et al.  Adaptation in clinical development plans and adaptive clinical trial designs , 2012 .

[43]  Tze Leung Lai,et al.  A New Approach to Modeling Covariate Effects and Individualization in Population Pharmacokinetics-Pharmacodynamics , 2005, Journal of Pharmacokinetics and Pharmacodynamics.

[44]  Susan Todd,et al.  Point and interval estimation following a sequential clinical trial , 1996 .

[45]  David L. DeMets,et al.  Design and analysis of group sequential tests based on the type I error spending rate function , 1987 .

[46]  P. Bauer,et al.  Adaptive Two Stage Designs and the Conditional Error Function , 1999 .

[47]  B. Schwartz,et al.  Intussusception among infants given an oral rotavirus vaccine. , 2001, The New England journal of medicine.

[48]  A. Wald.  Sequential Tests of Statistical Hypotheses , 1945 .

[49]  Gary Lorden,et al.  Asymptotic Efficiency of Three-Stage Hypothesis Tests , 1983 .

[50]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[51]  C Jennison,et al.  A group sequential t-test with updating of sample size , 2000 .

[52]  D O Scharfstein,et al.  The use of simulation and bootstrap in information-based group sequential studies. , 1998, Statistics in medicine.

[53]  Andrea J Cook,et al.  Challenges in the design and analysis of sequentially monitored postmarket safety surveillance evaluations using electronic observational health care data , 2012, Pharmacoepidemiology and drug safety.

[54]  G. Lorden.  2-SPRT'S and The Modified Kiefer-Weiss Problem of Minimizing an Expected Sample Size , 1976 .

[55]  C. Stein.  A Two-Sample Test for a Linear Hypothesis Whose Power is Independent of the Variance , 1945 .

[56]  R Simon,et al.  Designs for efficient clinical trials. , 1989, Oncology.

[57]  D. Harrington.  A class of rank test procedures for censored survival data , 1982 .

[58]  S Zacks,et al.  Cancer phase I clinical trials: efficient dose escalation with overdose control. , 1998, Statistics in medicine.

[59]  C. Burman,et al.  Are Flexible Designs Sound? , 2006, Biometrics.

[60]  D. Bates,et al.  Nonlinear mixed effects models for repeated measures data. , 1990, Biometrics.

[61]  Lingling Li,et al.  A conditional sequential sampling procedure for drug safety surveillance , 2009, Statistics in medicine.

[62]  T. Lai.  Nearly Optimal Sequential Tests of Composite Hypotheses , 1988 .

[63]  T. Lai.  Optimal Stopping and Sequential Tests which Minimize the Maximum Expected Sample Size , 1973 .

[64]  H. Robbins,et al.  Adaptive Design and Stochastic Approximation , 1979 .

[65]  Jay Bartroff,et al.  Efficient adaptive designs with mid‐course sample size adjustment in clinical trials , 2008, Statistics in medicine.

[66]  S. Day,et al.  Internal pilot studies for estimating sample size. , 1994, Statistics in medicine.

[67]  W. J. Hall,et al.  Unbiased estimation following a group sequential test , 1999 .

[68]  H. Müller,et al.  Hazard rate estimation under random censoring with varying kernels and bandwidths. , 1994, Biometrics.

[69]  P. Lachenbruch,et al.  The Design and Analysis of Sequential Clinical Trials (2nd ed.). , 1993 .

[70]  P. Armitage,et al.  Repeated Significance Tests on Accumulating Data , 1969 .

[71]  J. V. Ryzin,et al.  Nonparametric Bayesian Estimation of Survival Curves from Incomplete Observations , 1976 .

[72]  Tze Leung Lai,et al.  Resampling methods for confidence intervals in group sequential trials , 1998 .

[73]  Tze Leung Lai,et al.  Approximate Policy Optimization and Adaptive Control in Regression Models , 2005 .

[74]  Herman Chernoff,et al.  Sequential Tests for the Mean of a Normal Distribution , 1965 .

[75]  J. Kiefer.  General Equivalence Theory for Optimum Designs (Approximate Theory) , 1974 .

[76]  Christopher Jennison,et al.  Efficient group sequential designs when there are several effect sizes under consideration , 2006, Statistics in medicine.

[77]  W. Shih,et al.  A sample size adjustment procedure for clinical trials based on conditional power. , 2002, Biostatistics.

[78]  Anastasios A. Tsiatis,et al.  Exact confidence intervals following a group sequential trial: A comparison of methods , 1988 .

[79]  W. Shih.  Sample size re‐estimation – journey for a decade , 2001, Statistics in medicine.

[80]  Tze Leung Lai,et al.  Power, sample size and adaptation considerations in the design of group sequential clinical trials , 2003 .

[81]  J. Wittes,et al.  The Use of Interim Analysis for Sample Size Adjustment , 1993 .

[82]  B E Storer,et al.  Design and analysis of phase I clinical trials. , 1989, Biometrics.

[83]  J. Kiefer,et al.  Some Properties of Generalized Sequential Probability Ratio Tests , 1957 .

[84]  T. Lai,et al.  A Modification of schwarz's sequential likelihood ratio tests in multivariate sequential analysis , 1994 .

[85]  S. W. Roberts.  A Comparison of Some Control Chart Procedures , 1966 .

[86]  Y Tsong,et al.  Group sequential test strategies for superiority and non‐inferiority hypotheses in active controlled clinical trials , 2001, Statistics in medicine.

[87]  Larry Rubinstein,et al.  Randomized Phase II Designs , 2009, Clinical Cancer Research.

[88]  G. Dantzig.  On the Non-Existence of Tests of "Student's" Hypothesis Having Power Functions Independent of $\sigma$ , 1940 .

[89]  David Azriel,et al.  Optimal sequential designs in phase I studies , 2014, Comput. Stat. Data Anal..

[90]  R. Prentice.  Linear rank tests with right censored data , 1978 .

[91]  Benjamin Yakir,et al.  Inference about a secondary process following a sequential trial , 2003 .

[92]  L. Skovgaard.  NONLINEAR MODELS FOR REPEATED MEASUREMENT DATA. , 1996 .

[93]  T. Lai.  Likelihood Ratio Identities and Their Applications to Sequential Analysis , 2004 .

[94]  G. Elfving.  Optimum Allocation in Linear Regression Theory , 1952 .

[95]  Peter Armitage,et al.  16. The Design and Analysis of Sequential Clinical Trials , 1993 .

[96]  S. Kelsey,et al.  National Heart, Lung, and Blood Institute type II Coronary Intervention Study: design, methods, and baseline characteristics. , 1982, Controlled clinical trials.

[97]  B. Katzung.  Basic and Clinical Pharmacology , 1982 .

[98]  Anastasios A. Tsiatis,et al.  Group sequential designs for one-sided and two-sided hypothesis testing with provision for early stopping in favor of the null hypothesis , 1994 .

[99]  Tze Leung Lai,et al.  Confidence intervals for survival quantiles in the Cox regression model , 2006, Lifetime data analysis.

[100]  Jay Bartroff,et al.  Generalized Likelihood Ratio Statistics and Uncertainty Adjustments in Efficient Adaptive Design of Clinical Trials , 2008, 1105.4667.

[101]  Michael A. Proschan,et al.  Statistical Monitoring of Clinical Trials: A Unified Approach , 2006 .

[102]  K K Lan,et al.  The B-value: a tool for monitoring data. , 1988, Biometrics.

[103]  Margaret S. Pepe,et al.  Two-stage experimental designs : early stopping with a negative result , 1992 .

[104]  W. Lehmacher,et al.  Adaptive Sample Size Calculations in Group Sequential Trials , 1999, Biometrics.

[105]  E. Newlands,et al.  Phase I trial of temozolomide (CCRG 81045: M&B 39831: NSC 362856). , 1992, British Journal of Cancer.

[106]  William H. Press,et al.  Numerical recipes in C. The art of scientific computing , 1987 .

[107]  John Eales,et al.  Optimal two-sided group sequential tests , 1995 .

[108]  G. Lorden.  PROCEDURES FOR REACTING TO A CHANGE IN DISTRIBUTION , 1971 .

[109]  Tze Leung Lai,et al.  Bias correction and confidence intervals following sequential tests , 2006, math/0611677.

[110]  J. Herson,et al.  Predictive probability early termination plans for phase II clinical trials. , 1979, Biometrics.

[111]  D. Harron.  Clinical pharmacokinetics: Concepts and applications (2nd edn.) , 1989 .

[112]  A. Tsiatis,et al.  On the inefficiency of the adaptive design for monitoring clinical trials , 2003 .

[113]  Joseph F Heyse,et al.  Evaluating the safety of a rotavirus vaccine: the REST of the story , 2008, Clinical trials.

[114]  Tze Leung Lai,et al.  Confidence intervals in group sequential trials with random group sizes and applications to survival analysis , 2004 .

[115]  A Whitehead,et al.  Mid-trial design reviews for sequential clinical trials. , 2001, Statistics in medicine.

[116]  Tze Leung Lai,et al.  Information Bounds and Quick Detection of Parameter Changes in Stochastic Systems , 1998, IEEE Trans. Inf. Theory.

[117]  K. K. Lan,et al.  Stochastically curtailed tests in long–term clinical trials , 1982 .

[118]  Alexander M. Mood,et al.  A Method for Obtaining and Analyzing Sensitivity Data , 1948 .

[119]  B. Turnbull,et al.  Group Sequential Methods with Applications to Clinical Trials , 1999 .

[120]  E. Gehan.  A GENERALIZED WILCOXON TEST FOR COMPARING ARBITRARILY SINGLY-CENSORED SAMPLES. , 1965, Biometrika.

[121]  W Katherine Yih,et al.  Near real‐time vaccine safety surveillance with partially accrued data , 2011, Pharmacoepidemiology and drug safety.

[122]  M. Proschan,et al.  Designed extension of studies based on conditional power. , 1995 .

[123]  Michael Nguyen,et al.  The Food and Drug Administration's Post‐Licensure Rapid Immunization Safety Monitoring program: strengthening the federal vaccine safety enterprise , 2012, Pharmacoepidemiology and drug safety.

[124]  W. Hoeffding.  Lower Bounds for the Expected Sample Size and the Average Risk of a Sequential Procedure , 1960 .

[125]  J. Peto,et al.  Asymptotically Efficient Rank Invariant Test Procedures , 1972 .

[126]  Frank Bretz,et al.  Confirmatory Seamless Phase II/III Clinical Trials with Hypotheses Selection at Interim: General Concepts , 2006, Biometrical journal. Biometrische Zeitschrift.

[127]  Mark Chang,et al.  Adaptive Design Methods in Clinical Trials , 2006 .

[128]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[129]  Tze Leung Lai,et al.  Modified Haybittle–Peto group sequential designs for testing superiority and non‐inferiority hypotheses in clinical trials , 2006, Statistics in medicine.

[130]  A. Willsky,et al.  A generalized likelihood ratio approach to the detection and estimation of jumps in linear systems , 1976 .

[131]  A. Gould,et al.  Sample size re-estimation without unblinding for normally distributed outcomes with unknown variance , 1992 .

[132]  J. Kiefer,et al.  Sequential Decision Problems for Processes with Continuous Time Parameter Problems of Estimation , 1953 .

[133]  Christopher Jennison,et al.  Mid‐course sample size modification in clinical trials based on the observed treatment effect , 2002, Statistics in medicine.

[134]  Christopher Jennison,et al.  Statistical Approaches to Interim Monitoring of Medical Trials: A Review and Commentary , 1990 .

[135]  M. Pollak.  Optimal Detection of a Change in Distribution , 1985 .

[136]  C. F. Wu,et al.  Efficient Sequential Designs with Binary Data , 1985 .

[137]  Russell D. Wolfinger,et al.  Laplace's approximation for nonlinear mixed models. , 1993 .

[138]  Tze Leung Lai,et al.  Sequential design of phase II–III cancer trials , 2012, Statistics in medicine.

[139]  Tze Leung Lai,et al.  Clinical trial designs for testing biomarker-based personalized therapies , 2012, Clinical trials.

[140]  Tze Leung Lai,et al.  Efficient Adaptive Randomization and Stopping Rules in Multi-arm Clinical Trials for Testing a New Treatment , 2012, Sequential analysis.

[141]  T. Lai.  SEQUENTIAL ANALYSIS: SOME CLASSICAL PROBLEMS AND NEW CHALLENGES , 2001 .

[142]  P. O'Brien,et al.  A multiple testing procedure for clinical trials. , 1979, Biometrics.

[143]  Bradley P. Carlin,et al.  Bayesian Adaptive Methods for Clinical Trials , 2010 .

[144]  M Gu,et al.  Determination of power and sample size in the design of clinical trials with failure-time endpoints and interim analyses. , 1999, Controlled clinical trials.

[145]  Jianqing Fan.  On the Optimal Rates of Convergence for Nonparametric Deconvolution Problems , 1991 .

[146]  Tze Leung Lai,et al.  Asymptotic approximations for error probabilities of sequential or fixed sample size tests in exponential families , 1999 .

[147]  D. D. Hoff,et al.  Response rates, duration of response, and dose response effects in phase I studies of antineoplastics , 1991, Investigational New Drugs.

[148]  Ethan Reiner,et al.  Operating characteristics of the standard phase I clinical trial design , 1999 .

[149]  J. Sacks.  Asymptotic Distribution of Stochastic Approximation Procedures , 1958 .

[150]  Jay Bartroff,et al.  Approximate Dynamic Programming and Its Applications to the Design of Phase I Cancer Trials , 2010, 1011.6509.

[151]  Akifumi Yafune,et al.  A Use of Monte Carlo Integration for Population Pharmacokinetics with Multivariate Population Distribution , 1998, Journal of Pharmacokinetics and Biopharmaceutics.

[152]  Michael Woodroofe,et al.  Very Weak Expansions for Sequential Confidence Levels , 1986 .

[153]  L Fisher,et al.  Statistical Inference for Self‐Designing Clinical Trials with a One‐Sided Hypothesis , 1999, Biometrics.

[154]  H. Chernoff.  Sequential Analysis and Optimal Design , 1987 .

[155]  Andrew J Vickers,et al.  Setting the Bar in Phase II Trials: The Use of Historical Data for Determining “Go/No Go” Decision for Definitive Phase III Testing , 2007, Clinical Cancer Research.

[156]  Eric V. Slud,et al.  Two-Sample Repeated Significance Tests Based on the Modified Wilcoxon Statistic , 1982 .

[157]  Niels Keiding,et al.  Statistical Models Based on Counting Processes , 1993 .

[158]  Christopher Jennison,et al.  Adaptive and nonadaptive group sequential tests , 2006 .

[159]  Herman Chernoff,et al.  SEQUENTIAL TEST FOR THE MEAN OF A NORMAL DISTRIBUTION III (SMALL t) , 1965 .

[160]  Tze Leung Lai,et al.  A hybrid estimator in nonlinear and generalised linear mixed effects models , 2003 .

[161]  Robert L Davis,et al.  Real-Time Vaccine Safety Surveillance for the Early Detection of Adverse Events , 2007, Medical care.

[162]  Susan S Ellenberg,et al.  Evaluating the safety of new vaccines: summary of a workshop. , 2005, American journal of public health.

[163]  Mark Chang,et al.  Adaptive Design Theory and Implementation using SAS and R , 2007 .

[164]  Tze Leung Lai,et al.  BOOTSTRAP METHODS FOR TRUNCATED AND CENSORED DATA , 1996 .

[165]  X. Xiong,et al.  Testing Secondary Hypotheses Following Sequential Clinical Trials , 2000, Biometrics.

[166]  S. Pocock.  Group sequential methods in the design and analysis of clinical trials , 1977 .

[167]  F. J. Anscombe,et al.  Large-sample theory of sequential estimation , 1949, Mathematical Proceedings of the Cambridge Philosophical Society.

[168]  P. Hall.  The Bootstrap and Edgeworth Expansion , 1992 .

[169]  David Bock,et al.  A review and discussion of prospective statistical surveillance in public health , 2003 .

[170]  D. Cox.  Regression Models and Life-Tables , 1972 .

[171]  P. Bauer,et al.  Evaluation of experiments with adaptive interim analyses. , 1994, Biometrics.

[172]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[173]  Ron Brookmeyer,et al.  A Confidence Interval for the Median Survival Time , 1982 .

[174]  Jing Ning,et al.  Using short‐term response information to facilitate adaptive randomization for survival clinical trials , 2009, Statistics in medicine.

[175]  L. Haines,et al.  Bayesian Optimal Designs for Phase I Clinical Trials , 2003, Biometrics.

[176]  R. Simon,et al.  Optimal two-stage designs for phase II clinical trials. , 1989, Controlled clinical trials.

[177]  Benjamin Schwartz,et al.  The first rotavirus vaccine and intussusception: epidemiological studies and policy decisions. , 2003, The Journal of infectious diseases.

[178]  Peter F Thall,et al.  Seamlessly Expanding a Randomized Phase II Trial to Phase III , 2002, Biometrics.

[179]  Tze Leung Lai,et al.  Futility stopping in clinical trials , 2012 .

[180]  M. Kulldorff,et al.  A Maximized Sequential Probability Ratio Test for Drug and Vaccine Safety Surveillance , 2011 .

[181]  Michael Woodroofe,et al.  Estimation after sequential testing: A simple approach for a truncated sequential probability ratio test , 1992 .

[182]  Tze Leung Lai,et al.  REPEATED SIGNIFICANCE TESTING WITH CENSORED RANK STATISTICS IN INTERIM ANALYSIS OF CLINICAL TRIALS , 1998 .

[183]  Haipeng Xing,et al.  Sequential Change-Point Detection When the Pre- and Post-Change Parameters are Unknown , 2010 .

[184]  M. Christian,et al.  Phase I clinical trial design in cancer drug development. , 2000, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[185]  P. Armitage.  Sequential Medical Trials , 1961, Biomedicine / [publiee pour l'A.A.I.C.I.G.].

[186]  Nancy Flournoy,et al.  Random Walks for Quantile Estimation , 1994 .

[187]  J. Whitehead,et al.  A FORTRAN program for the design and analysis of sequential clinical trials. , 1983, Computers and biomedical research, an international journal.

[188]  T R Fleming,et al.  Symmetric group sequential test designs. , 1989, Biometrics.

[189]  J O'Quigley,et al.  Continual reassessment method: a practical design for phase 1 clinical trials in cancer. , 1990, Biometrics.

[190]  J. Haybittle,et al.  Repeated assessment of results in clinical trials of cancer treatment. , 1971, The British journal of radiology.

[191]  Christopher Jennison,et al.  An improved method for deriving optimal one-sided group sequential tests , 1992 .

[192]  Richard A. Johnson.  Asymptotic Expansions Associated with Posterior Distributions , 1970 .

[193]  Andrea J Cook,et al.  Statistical approaches to group sequential monitoring of postmarket safety surveillance data: current state of the art for use in the Mini‐Sentinel pilot , 2012, Pharmacoepidemiology and drug safety.

[194]  Robert T. Chen,et al.  Active Surveillance of Vaccine Safety: A System to Detect Early Signs of Adverse Events , 2005, Epidemiology.

[195]  T. Lai.  ON OPTIMAL STOPPING PROBLEMS IN SEQUENTIAL HYPOTHESIS TESTING , 1997 .

[196]  James M. Robins,et al.  Semiparametric Efficiency and its Implication on the Design and Analysis of Group-Sequential Studies , 1997 .

[197]  M. Pike,et al.  Design and analysis of randomized clinical trials requiring prolonged observation of each patient. II. analysis and examples. , 1977, British Journal of Cancer.

[198]  Tze Leung Lai,et al.  Rank Tests Based on Censored Data and Their Sequential Analogues , 1991 .

[199]  Richard Platt,et al.  The U.S. Food and Drug Administration's Mini‐Sentinel program: status and direction , 2012, Pharmacoepidemiology and drug safety.

[200]  G. Schwarz.  Asymptotic Shapes of Bayes Sequential Testing Regions , 1962 .

[201]  D. Spiegelhalter,et al.  Monitoring clinical trials: conditional or predictive power? , 1986, Controlled clinical trials.

[202]  D. S. Bayard.  A forward method for optimal stochastic nonlinear and adaptive control , 1991 .

[203]  Tze Leung Lai,et al.  Weak Convergence of Time-Sequential Censored Rank Statistics with Applications to Sequential Testing in Clinical Trials , 1991 .

[204]  Tze Leung Lai,et al.  Nonparametric estimation in nonlinear mixed effects models , 2003 .

[205]  L D Fisher,et al.  Self-designing clinical trials. , 1998, Statistics in medicine.

[206]  Ying Yuan,et al.  Bayesian Model Averaging Continual Reassessment Method in Phase I Clinical Trials , 2009 .