Statistical performance of group sequential methods for observational post-licensure medical product safety surveillance: A simulation study

In order to improve post-licensure drug and vaccine safety surveillance, new systems are being developed that prospectively monitor observational health care data from large health plans. Continuous sequential testing has been proposed in this setting to facilitate rapid detection, but group sequential methods commonly used in randomized clinical trials (RCTs) have received less consideration. We propose a group sequential approach tailored for safety and to account for complications like confounding that arise in this non-randomized setting and thus have not been previously examined in RCTs. For comparability with prior continuous monitoring applications, we use a likelihood ratio statistic and historical controls. We compute sequential boundaries using Monte Carlo simulation and show how they can accommodate unequal between-test sample sizes and changes in confounder distributions among accruing subjects over time. We evaluate via simulation the performance of this approach across sequential designs suited for safety and not previously addressed by simulation studies evaluating RCT boundaries. Such designs include much higher frequency testing and designs that employ early conservatism followed by frequent testing. Contrary to prior RCT simulations, we found major differences in the average time-to-surveillanceend and overall power. We apply this methodology to safety data on a new pediatric combination vaccine.