On the falsifiability of the nested clade phylogeographic analysis method

The nested clade phylogeographic analysis (NCPA) method was developed to handle and interpret phylogeographic data sets (Templeton et al. 1995). Panchal & Beaumont (2007) have recently developed software to automate the NCPA procedure. Using simulations, they demonstrate that this method returns a very large proportion (> 80%) of false-positives in the simplest case possible — a single locus and random-mating populations. In view of this new finding, my opinion was that the method should no longer be used until it has been more thoroughly and critically evaluated (Petit 2008). Garrick et al. (2008) argue instead that the NCPA method could still be useful to generate ‘plausible hypotheses’, not in isolation but as corroborating evidence. They suggest that in this case, its benefits should counter the high false-positive rates, given the lack of ‘computationally feasible substitute’. While I agree that it can be difficult to draw the line between the expected rigor and the necessary pragmatism needed in historical sciences, the absence of an alternative method does not strike me as an appropriate argument. Garrick et al. (2008) further recall that the inclusion of several genes can substantially improve accuracy when reconstructing organismal history from molecular data, a rather uncontroversial statement. They believe that as additional loci are added, the bias in type I error will be reduced in NCPA analyses. Instead of arguing otherwise, I suggest that this particular hypothesis be tested using appropriate simulations with multiple loci, which should be facilitated by the new automated NCPA procedure (Panchal 2007). In the meantime, the results of Panchal & Beaumont (2007) convince me that reputable journals should (i) discourage the use of the NCPA method for single locus data sets (which represents the vast majority of surveys published to date), and (ii) still be suspicious of NCPA analyses based on multiple loci (until formal analyses or simulations demonstrate that NCPA works indeed better with multiple loci than with a single locus). Like Garrick et al. (2008), I see with great interest the rise in the number of collaborations between phylogeographers and Earth scientists. Clearly, this can lead to well-defined prior expectations when studying species evolutionary histories. However, I do not see how developing well-defined prior hypotheses could help rescue the NCPA method if it is proved to be misleading. In my commentary, I also pointed out a potential problem with the NCPA permutation test (see also Petit & Grivet 2002). My objective was not to discuss in general terms whether one particular type of sampling (individual vs. population) is better than the other. Instead, I wanted to recall a common problem with spatial statistics, which emerges from the statistical nonindependence of individuals when these are sampled in clusters (the populations) rather than at random in the frame of a phylogeographic survey. To sum up, I support the Garrick et al. (2008) conclusion that a constructive course of action is to encourage the validation of the NCPA. However, I suggest that such validation should not be indirect, as a ‘part of a multifaceted battery of analyses’ but that it should instead involve careful simulations with known a priori parameters. While a ‘corroborative approach’ is eventually needed in any type of historical investigation, I contend that it is not appropriate for the (technical) validation of a formal statistical method of data analysis.