Problems modelling behavioural variation across Western North American Indian societies

Downloaded from http://rspb.royalsocietypublishing.org/ on March 16, 2016 rspb.royalsocietypublishing.org Comment Cite this article: Towner MC, Grote MN, Borgerhoff Mulder M. 2016 Problems modelling behavioural variation across Western North American Indian societies. Proc. R. Soc. B http://dx.doi.org/10.1098/rspb.2015.2184 Received: 10 September 2015 Accepted: 8 December 2015 Author for correspondence: Mary C. Towner e-mail: mary.towner@okstate.edu The accompanying reply can be viewed at http://dx.doi.org/10.1098/rspb.2016.0177. Problems modelling behavioural variation across Western North American Indian societies Mary C. Towner 1 , Mark N. Grote 2 and Monique Borgerhoff Mulder 2 Department of Integrative Biology, Oklahoma State University, Stillwater, OK 74078, USA Department of Anthropology, University of California, Davis, CA 95616, USA MCT, 0000-0002-0784-1860 Mathew & Perreault [1] analyse cross-cultural data from the Western North Amer- ican Indian (WNAI) dataset [2] in order to compare ‘the relative effect of environment and cultural history’ on behavioural variation across 172 societies. This endeavour is inspired by many other evolutionary studies of human cultural variation [3–7]. Mathew and Perreault conclude that ‘social learning operating over multiple generations [is] the main mode by which humans acquire their behaviour’ (p. 5). Our own investigation of cultural macroevolution in the WNAI [8] motivated us to attempt to reconstruct their analyses. we found their paper to be undermined by questionable analytical choices, and computational and data-handling pro- blems. We draw this conclusion having used the information in the Methods and electronic supplementary material S1, S3, S4 and S6 in [1] to recreate those parts of their study that we were able to. In this commentary, we present the results of our examination and detail the serious methodological flaws that lead us to con- clude that a complete re-analysis is required by Mathew and Perreault. We also comment briefly on their conceptual schema, which, in trying to find ‘the main mode of human adaptation’ [1], appears to set cultural transmission (i.e. social learning) in opposition to environmental adaptation. Mathew and Perreault use logistic regression to model 457 present/absent be- havioural traits as a function of three dimensions—E (local ecological conditions), P ( phylogenetic or linguistic distance to other societies) and S (spatial or geo- graphical distance to other societies). To judge the relative importance of the E, P and S classes of predictors, Mathew and Perreault compare sums of absolute values of regression coefficients across classes, for the best model of each behav- ioural trait. The ‘summed absolute values’ metric is used for various purposes in model and feature selection [9,10]. The metric is problematic here, however, because it compounds statistical signal with different sizes of the E, P and S classes. To demonstrate, consider a null case in which none of the predictors in E, P or S are related to a trait and the regression coefficients resemble stochastic noise. The analyst must understand how a statistical metric would behave in such a case and choose an inference procedure that reliably distinguishes null from non-null cases. For concreteness, assume that the coefficients share a common Gaussian distribution with mean zero and variance s 2 . The absolute value of a coefficient b then has expectation ð2 s 2 = p Þ 1=2 , and for a class containing M predictors the summed absolute values have expectation P E½ M . The null expectation therefore scales linearly with i¼1 j b i j ¼ Mð2 s = p Þ class size M, and larger classes of predictors will appear to have greater relative importance based on representation alone. Although model selection criteria such as AIC (discussed below) include a penalty for the number of predictors in a model, this does not mitigate the confounding effects of class size [11]. To see how summed absolute values depend on the number of predictors contributing to the sum, consider fig. 3 of [1], which appears to show a time- depth effect of language. The numbers of predictors in phylogenetic classes, from the Level 8 class down to the Level 3 class, follow a descending sequence: M ¼ 8, 8, 6, 4, 4, 3 (see electronic supplementary material S7 in [1]). This sequence is mirrored by the stair-step feature of the overall effect of phylogeny and is & 2016 The Author(s) Published by the Royal Society. All rights reserved.