Simultaneous confidence intervals for comparing margins of multivariate binary data

In many applications two groups are compared simultaneously on several correlated binary variables for a more comprehensive assessment of group differences. Although the response is multivariate, the main interest is in comparing the marginal probabilities between the groups. Estimating the size of these differences under strong error control allows for a better evaluation of effects than can be provided by multiplicity adjusted P-values. Simultaneous confidence intervals for the differences in marginal probabilities are developed through inverting the maximum of correlated Wald, score or quasi-score statistics. Taking advantage of the available correlation information leads to improvements in the joint coverage probability and power compared to straightforward Bonferroni adjustments. Estimating the correlation under the null is also explored. While computationally complex even in small dimensions, it does not result in marked improvements. Based on extensive simulation results, a simple approach that uses univariate score statistics together with their estimated correlation is proposed and recommended. All methods are illustrated using data from a vaccine trial that investigated the incidence of four pre-specified adverse events between two groups and with data from the General Social Survey.

[1]  Alastair Scott,et al.  Quick Simultaneous Confidence Intervals for Multinomial Proportions , 1987 .

[2]  A. Genz,et al.  Computation of Multivariate Normal and t Probabilities , 2009 .

[3]  J. Heyse,et al.  Use of the false discovery rate for evaluating clinical safety data , 2004, Statistical methods in medical research.

[4]  Ori Davidov,et al.  Order-Restricted Inference for Multivariate Binary Data With Application to Toxicology , 2011, Journal of the American Statistical Association.

[5]  V. P. Bhapkar,et al.  Multiple comparisons of matched proportions , 1976 .

[6]  P. A. R. Koopman,et al.  Confidence intervals for the ratio of two binomial proportions , 1984 .

[7]  W. Piegorsch,et al.  Large-sample pairwise comparisons among multinomial proportions with an application to analysis of mutant spectra , 2001 .

[8]  D. Boos.  On Generalized Score Tests , 1992 .

[9]  Markku Nurminen,et al.  In Defence of Score Intervals for Proportions and their Differences , 2011 .

[10]  L. A. Goodman.  On Simultaneous Confidence Intervals for Multinomial Proportions , 1965 .

[11]  M Lefkopoulou,et al.  Global tests for multiple binary outcomes. , 1993, Biometrics.

[12]  N L Geller,et al.  The analysis of multiple endpoints in clinical trials. , 1987, Biometrics.

[13]  J. Glaz,et al.  Simultaneous confidence intervals for multinomial proportions , 1999 .

[14]  Joseph B. Lang,et al.  Maximum likelihood methods for a generalized class of log-linear models , 1996 .

[15]  Alan Agresti,et al.  Simple and Effective Confidence Intervals for Proportions and Differences of Proportions Result from Adding Two Successes and Two Failures , 2000 .

[16]  Y. Benjamini,et al.  False Discovery Rate–Adjusted Multiple Confidence Intervals for Selected Parameters , 2005 .

[17]  S. Young,et al.  p Value Adjustments for Multiple Tests in Multivariate Binomial Models , 1989 .

[18]  Joseph B. Lang,et al.  Multinomial-Poisson homogeneous models for contingency tables , 2003 .

[19]  G Wassmer,et al.  Procedures for two-sample comparisons with multiple endpoints controlling the experimentwise error rate. , 1991, Biometrics.

[20]  Christian Ritz,et al.  A versatile method for confirmatory evaluation of the effects of a covariate in multiple models , 2012 .

[21]  H. Keselman,et al.  Multiple Comparison Procedures , 2005 .

[22]  Joseph Glaz,et al.  Simultaneous Confidence Intervals and Sample Size Determination for Multinomial Proportions , 1995 .

[23]  P. Westfall.  Simultaneous small-sample multivariate Bernoulli confidence intervals. , 1985, Biometrics.

[24]  Djalil CHAFAÏ,et al.  Confidence Regions for the Multinomial Parameter With Small Sample Size , 2008, 0805.1971.

[25]  Bahjat F. Qaqish,et al.  A family of multivariate binary distributions for simulating correlated binary variables with specified marginal means and correlations , 2003 .

[26]  Scott M Berry,et al.  Accounting for Multiplicities in Assessing Drug Safety: A Three‐Level Hierarchical Mixture Model , 2004, Biometrics.

[27]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[28]  R. Tiwari,et al.  A Likelihood Ratio Test Based Method for Signal Detection With Application to FDA’s Drug Safety Data , 2011 .

[29]  Alan Agresti,et al.  Multivariate tests comparing binomial probabilities, with application to safety studies for drugs , 2005 .

[30]  D. Mehrotra,et al.  Flagging clinical adverse experiences: reducing false discoveries without materially compromising power for detecting true signals , 2012, Statistics in medicine.

[31]  T. Hothorn,et al.  Simultaneous Inference in General Parametric Models , 2008, Biometrical journal. Biometrische Zeitschrift.

[32]  Bernhard Klingenberg,et al.  Simultaneous score confidence bounds for risk differences in multiple comparisons to a control , 2012, Comput. Stat. Data Anal..

[33]  A. Tamhane,et al.  Multiple Comparison Procedures , 1989 .