Use of the false discovery rate for evaluating clinical safety data

Clinical adverse experience (AE) data are routinely evaluated using between group P values for every AE encountered within each of several body systems. If the P values are reported and interpreted without multiplicity considerations, there is a potential for an excess of false positive findings. Procedures based on confidence interval estimates of treatment effects have the same potential for false positive findings as P value methods. Excess false positive findings can needlessly complicate the safety profile of a safe drug or vaccine. Accordingly, we propose a novel method for addressing multiplicity in the evaluation of adverse experience data arising in clinical trial settings. The method involves a two-step application of adjusted P values based on the Benjamini and Hochberg1 false discovery rate (FDR). Data from three moderate to large vaccine trials are used to illustrate our proposed ‘Double FDR’ approach, and to reinforce the potential impact of failing to account for multiplicity. This work was in collaboration with the late Professor John W. Tukey who coined the term ‘Double FDR’.