Evaluating different approaches that test whether microbial communities have the same structure

As microbial ecology investigations have progressed from descriptive characterizations of a community to hypothesis-driven ecological research, a number of different statistical techniques have been developed to describe and compare the structure of microbial communities. Thus far, these methods have only been evaluated using 16S rRNA gene sequence data obtained from incomplete characterizations of microbial communities. In this investigation, simulations were designed to test the statistical power of different methods to differentiate between communities with known memberships and structures. These simulations revealed three important results that affect how the results of the tests are interpreted. First, ∫-LIBSHUFF, TreeClimber, UniFrac, analysis of molecular variance (AMOVA) and homogeneity of molecular variance (HOMOVA) compare the structure of communities and not just their memberships. Second, ∫-LIBSHUFF is unable to detect cases when one community structure is a subset of another. Third, AMOVA determines whether the genetic diversity within two or more communities is greater than their pooled genetic diversity, and HOMOVA determines whether the amount of genetic diversity in each community is significantly different. ∫-LIBSHUFF, TreeClimber and UniFrac lump these and other factors together when performing their analysis making it difficult to discern the nature of the differences that are detected between communities. These findings demonstrate that when correctly employed, the current statistical toolbox has the ability to address specific ecological questions concerning the differences between microbial communities.