Dismantling the Mantel tests

Summary 1. The simple and partial Mantel tests are routinely used in many areas of evolutionary biology to assess the significance of the association between two or more matrices of distances relative to the same pairs of individuals or demes. Partial Mantel tests rather than simple Mantel tests are widely used to assess the relationship between two variables displaying some form of structure. 2. We show that contrary to a widely shared belief, partial Mantel tests are not valid in this case, and their bias remains close to that of the simple Mantel test. 3. We confirm that strong biases are expected under a sampling design and spatial correlation parameter drawn from an actual study. 4. The Mantel tests should not be used in case autocorrelation is suspected in both variables compared under the null hypothesis. We outline alternative strategies. The R code used for our computer simulations is distributed as supporting material.

[1]  D Hémon,et al.  Assessing the significance of the correlation between two spatial processes. , 1989, Biometrics.

[2]  Robert R. Sokal,et al.  An investigation of three-matrix permutation tests , 1992 .

[3]  Hoon Kim,et al.  Monte Carlo Statistical Methods , 2000, Technometrics.

[4]  J. Kingman.  Uses of Exchangeability , 1978 .

[5]  L. Waits,et al.  Putting the ‘landscape’ in landscape genetics , 2007, Heredity.

[6]  Peter J. Diggle,et al.  An Introduction to Model-Based Geostatistics , 2003 .

[7]  E. Dietz.  Permutation Tests for Association Between Two Distance Matrices , 1983 .

[8]  P. Legendre,et al.  Comparison tests for dendrograms: A comparative evaluation , 1995 .

[9]  L. Harmon,et al.  POOR STATISTICAL PERFORMANCE OF THE MANTEL TEST IN PHYLOGENETIC COMPARATIVE ANALYSES , 2010, Evolution; international journal of organic evolution.

[10]  R. Sokal,et al.  Multiple regression and correlation extensions of the mantel test of matrix correspondence , 1986 .

[11]  N. L. Johnson,et al.  Multivariate Analysis , 1958, Nature.

[12]  François Rousset,et al.  PARTIAL MANTEL TESTS: REPLY TO CASTELLANO AND BALLETTO , 2002 .

[13]  S. Castellano,et al.  IS THE PARTIAL MANTEL TEST INADEQUATE? , 2002, Evolution; international journal of organic evolution.

[14]  C. P. Robert.  James E. Gentle: Computational statistics (Statistics and Computing Series) , 2011, Stat. Comput..

[15]  P J Diggle,et al.  Spatial modelling and the prediction of Loa loa risk: decision making under uncertainty , 2007, Annals of tropical medicine and parasitology.

[16]  F. Rousset,et al.  ARE PARTIAL MANTEL TESTS ADEQUATE? , 2001, Evolution; international journal of organic evolution.

[17]  Monique Borgerhoff Mulder,et al.  Comparative Methods for Studying Cultural Trait Evolution: A Simulation Study , 2006 .

[18]  Raymond J. Dezzani,et al.  Statistical approaches in landscape genetics: an evaluation of methods for linking landscape and genetic data , 2009 .

[19]  Samuel A. Cushman,et al.  Gene Flow in Complex Landscapes: Testing Multiple Hypotheses with Causal Modeling , 2006, The American Naturalist.

[20]  Jean-Michel Marin,et al.  Approximate Bayesian computational methods , 2011, Statistics and Computing.

[21]  H. Wackernagle,et al.  Multivariate geostatistics: an introduction with applications , 1998 .

[22]  S. Cushman,et al.  Spurious correlations and inference in landscape genetics , 2010, Molecular ecology.

[23]  A. Gelfand,et al.  Handbook of spatial statistics , 2010 .

[24]  M. Fortin,et al.  Comparison of the Mantel test and alternative approaches for detecting complex multivariate relationships in the spatial analysis of genetic data , 2010, Molecular ecology resources.

[25]  Woojoo Lee,et al.  Modifications of REML algorithm for HGLMs , 2012, Stat. Comput..

[26]  John A. Nelder,et al.  Likelihood Inference for Models with Unobservables: Another View , 2009 .

[27]  Robert R. Sokal,et al.  Testing Statistical Significance of Geographic Variation Patterns , 1979 .

[28]  N. Mantel.  The detection of disease clustering and a generalized regression approach. , 1967, Cancer research.

[29]  Andrea Cerioli,et al.  Testing Mutual Independence Between Two Discrete‐Valued Spatial Processes: A Correction to Pearson Chi‐Squared , 2002, Biometrics.

[30]  P. Legendre,et al.  ANALYZING BETA DIVERSITY: PARTITIONING THE SPATIAL VARIATION OF COMMUNITY COMPOSITION DATA , 2005 .

[31]  P. Clifford,et al.  Modifying the t test for assessing the correlation between two spatial processes , 1993 .

[32]  P. Clifford,et al.  Testing association between spatial processes , 1991 .

[33]  G. J. G. Upton,et al.  Spatial data Analysis by Example , 1985 .

[34]  J. Diniz‐Filho,et al.  Spatial autocorrelation and red herrings in geographical ecology , 2003 .

[35]  Pierre Legendre,et al.  Comparison of permutation methods for the partial correlation and partial mantel tests , 2000 .

[36]  Peter J. Diggle,et al.  Detecting dependence between marks and locations of marked point processes , 2004 .