Marginal Effects in Interaction Models: Determining and Controlling the False Positive Rate

When a researcher suspects that the marginal effect of x on y varies with z , a common approach is to plot ∂ y / ∂ x at different values of z along with a pointwise confidence interval generated using the procedure described in Brambor, Clark, and Golder to assess the magnitude and statistical significance of the relationship. Our article makes three contributions. First, we demonstrate that the Brambor, Clark, and Golder approach produces statistically significant findings when ∂ y / ∂ x = 0 at a rate that can be many times larger or smaller than the nominal false positive rate of the test. Second, we introduce the interactionTest software package for R to implement procedures that allow easy control of the false positive rate. Finally, we illustrate our findings by replicating an empirical analysis of the relationship between ethnic heterogeneity and the number of political parties from Comparative Political Studies.

[1]  Erich L. Lehmann.  Testing Multiparameter Hypotheses , 1952 .

[2]  E. L. Lehmann,et al.  A Theory of Some Multiple Decision Problems. II , 1957 .

[3]  Z. Šidák.  Rectangular Confidence Regions for the Means of Multivariate Normal Distributions , 1967 .

[4]  V. Barnett,et al.  Applied Linear Statistical Models , 1975 .

[5]  S. Holm.  A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[6]  Roger L. Berger,et al.  Multiparameter Hypothesis Testing and Acceptance Sampling , 1982 .

[7]  A. Cohen,et al.  Hypothesis Testing for Marginal Probabilities in a 2 × 2 × 2 Contingency Table with Conditional Independence , 1983 .

[8]  Y. Hochberg.  A sharper Bonferroni procedure for multiple tests of significance , 1988 .

[9]  D. Rom.  A sequentially rejective test procedure based on a modified Bonferroni inequality , 1990 .

[10]  J. Shaffer.  Multiple Hypothesis Testing , 1995 .

[11]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[12]  Roger L. Berger,et al.  Likelihood ratio tests and intersection-union tests , 1997 .

[13]  Octavio Amorim Neto,et al.  Electoral institutions,cleavage structures,and the number of parties , 1997 .

[14]  Kenneth A. Schultz.  Do Democratic Institutions Constrain or Inform? Contrasting Two Institutional Perspectives on Democracy and War , 1999, International Organization.

[15]  Jason Wittenberg,et al.  Making the Most Of Statistical Analyses: Improving Interpretation and Presentation , 2000 .

[16]  P. Sen,et al.  Constrained Statistical Inference: Inequality, Order, and Shape Restrictions , 2001 .

[17]  Y. Benjamini,et al.  THE CONTROL OF THE FALSE DISCOVERY RATE IN MULTIPLE TESTING UNDER DEPENDENCY , 2001 .

[18]  C. Ai,et al.  Interaction terms in logit and probit models , 2003 .

[19]  Bear F. Braumoeller.  Hypothesis Testing and Multiplicative Interaction Terms , 2004, International Organization.

[20]  Y. Benjamini,et al.  False Discovery Rate–Adjusted Multiple Confidence Intervals for Selected Parameters , 2005 .

[21]  Matt Golder,et al.  Rehabilitating Duverger’s Theory , 2006 .

[22]  H. Abdi.  The Bonferonni and Šidák Corrections for Multiple Comparisons , 2006 .

[23]  Thomas Brambor,et al.  Understanding Interaction Models: Improving Empirical Analyses , 2006, Political Analysis.

[24]  Cindy D. Kam,et al.  Modeling and Interpreting Interactive Hypotheses in Regression Analysis , 2007 .

[25]  A. Reiner-Benaim.  FDR Control by the BH Procedure for Two‐Sided Correlated Tests with Implications to Gene Expression Data Analysis , 2007, Biometrical journal. Biometrische Zeitschrift.

[26]  George Casella,et al.  Statistical Inference Second Edition , 2007 .

[27]  Y. Benjamini.  Discovering the false discovery rate , 2010 .

[28]  William D. Berry,et al.  Testing for Interaction in Binary Logit and Probit Models: Is a Product Term Essential? , 2010 .

[29]  William D. Berry,et al.  Improving Tests of Theories Positing Interaction , 2012 .

[30]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[31]  Carlisle Rainey,et al.  Arguing for a Negligible Effect , 2014 .

[32]  J. Esarey,et al.  Corrigendum to “Marginal Effects in Interaction Models: Determining and Controlling the False Positive Rate” , 2020 .