Bayesian significance testing and multiple comparisons from MCMC outputs

This article proposes a Bayesian method to directly evaluate and test hypotheses in multiple comparisons. Transformation and integration over the coordinates relevant to the hypothesis are shown to enable us to directly test the hypotheses expressed as a linear equation of a parameter vector, given a linear constraint. When the conditional posterior distribution of the parameter vector we are interested in is the multivariate normal distribution, the proposed method can be applied to calculate the p-value of hypotheses pertaining to the parameters in any complex model such as generalized linear mixed effect models with latent variables, by using outputs from Markov chain Monte Carlo (MCMC) methods. Further, the proposed testing can be implemented without prior information. Some applications are presented, and the simulation results are provided to compare the powers of this method with those of other methods of conventional multiple comparisons. Simulation studies have shown that the proposed method is valid for multiple comparisons under nonequivalent variances and mean comparisons in latent variable modeling with categorical variables.

[1]  J. Berger,et al.  Testing a Point Null Hypothesis: The Irreconcilability of P Values and Evidence , 1987 .

[2]  J. Albert.  Bayesian Estimation of Normal Ogive Item Response Curves Using Gibbs Sampling , 1992 .

[3]  E. Lehmann.  Testing Statistical Hypotheses , 1960 .

[4]  Takahiro Hoshino,et al.  A Bayesian propensity score adjustment for latent variable modeling and MCMC algorithm , 2008, Comput. Stat. Data Anal..

[5]  M. Tanner.  Tools for statistical inference: methods for the exploration of posterior distributions and likeliho , 1994 .

[6]  D. Berry,et al.  Bayesian multiple comparisons using dirichlet process priors , 1998 .

[7]  K. Gabriel,et al.  On closed testing procedures with special reference to ordered analysis of variance , 1976 .

[8]  A Propensity Score Adjustment for Multiple Group Structural Equation Modeling , 2006 .

[9]  A. Tamhane,et al.  Multiple Comparison Procedures , 1989 .

[10]  W. Meredith.  Measurement invariance, factor analysis and factorial invariance , 1993 .

[11]  D. Newman,et al.  THE DISTRIBUTION OF RANGE IN SAMPLES FROM A NORMAL POPULATION, EXPRESSED IN TERMS OF AN INDEPENDENT ESTIMATE OF STANDARD DEVIATION , 1939 .

[12]  Philip H. Ramsey.  Power Differences between Pairwise Multiple Comparisons , 1978 .

[13]  A. Zellner.  An Introduction to Bayesian Inference in Econometrics , 1971 .

[14]  Z. Šidák.  Rectangular Confidence Regions for the Means of Multivariate Normal Distributions , 1967 .

[15]  D. Dunson,et al.  Bayesian latent variable models for clustered mixed outcomes , 2000 .

[16]  K. Jöreskog.  Simultaneous factor analysis in several populations , 1971 .

[17]  L. Ryan,et al.  Latent Variable Models for Mixed Discrete and Continuous Outcomes , 1997 .

[18]  K. Jöreskog.  A general method for analysis of covariance structures , 1970 .

[19]  H. V. Dijk,et al.  A Bayesian analysis of the unit root in real exchange rates , 1991 .

[20]  Richard J. Meinhold,et al.  Robustification of Kalman Filter Models , 1989 .

[21]  R. A. Waller,et al.  A Bayes Rule for the Symmetric Multiple Comparisons Problem , 1969 .

[22]  William Mendenhall,et al.  Introduction to Probability and Statistics , 1961, The Mathematical Gazette.

[23]  S. Holm.  A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[24]  S. Chib,et al.  Bayesian analysis of binary and polychotomous response data , 1993 .

[25]  Sylvia Richardson,et al.  Markov Chain Monte Carlo in Practice , 1997 .

[26]  D. B. Duncan.  t TESTS AND INTERVALS FOR COTIPARISONS SUGGESTED BY THE DATA , 1975 .

[27]  G. C. Tiao,et al.  Bayesian inference in statistical analysis , 1973 .

[28]  D. Lindley.  A STATISTICAL PARADOX , 1957 .

[29]  P. Bentler.  Some contributions to efficient statistics in structural models: Specification and estimation of moment structures , 1983 .

[30]  J. Berger.  Statistical Decision Theory and Bayesian Analysis , 1988 .

[31]  D. Andrews.  The Large Sample Correspondence between Classical Hypothesis Tests and Bayesian Posterior Odds Tests , 1994 .

[32]  M. Keuls,et al.  The use of the „studentized range” in connection with an analysis of variance , 1952, Euphytica.

[33]  R. Welsch.  Stepwise Multiple Comparison Procedures , 1977 .

[34]  L. Tierney.  Markov Chains for Exploring Posterior Distributions , 1994 .

[35]  Dongcheol Kim,et al.  A Bayesian Significance Test of the Stationarity of Regression Parameters , 1991 .

[36]  Martin A. Tanner,et al.  Calculating the content and boundary of the highest posterior density region via data augmentation , 1990 .

[37]  J. Hogan,et al.  Bayesian Factor Analysis for Spatially Correlated Data, With Application to Summarizing Area-Level Material Deprivation From Census Data , 2004 .

[38]  Sylvia Richardson,et al.  Markov chain concepts related to sampling algorithms , 1995 .

[39]  K. Gabriel,et al.  A Study of the Powers of Several Methods of Multiple Comparisons , 1975 .

[40]  S. Chib,et al.  Analysis of multivariate probit models , 1998 .