On Sex, Evolution, and the Multiplicative Weights Update Algorithm

We consider a recent innovative theory by Chastain et al. on the role of sex in evolution [10]. In short, the theory suggests that the evolutionary process of gene recombination implements the celebrated multiplicative weights updates algorithm (MWUA). They prove that the population dynamics induced by sexual reproduction can be precisely modeled by genes that use MWUA as their learning strategy in a particular coordination game. The result holds in the environments of weak selection, under the assumption that the population frequencies remain a product distribution. We revisit the theory, eliminating both the requirement of weak selection and any assumption on the distribution of the population. Removing the assumption of product distributions is crucial, since as we show, this assumption is inconsistent with the population dynamics. We show that the marginal allele distributions induced by the population dynamics precisely match the marginals induced by a multiplicative weights update algorithm in this general setting, thereby affirming and substantially generalizing these earlier results. We further revise the implications for convergence and utility or fitness guarantees in coordination games. In contrast to the claim of Chastain et al.[10], we conclude that the sexual evolutionary dynamics does not entail any property of the population distribution, beyond those already implied by convergence.

[1]  C C Li Increment of average fitness for multiple alleles. , 1969, Proceedings of the National Academy of Sciences of the United States of America.

[2]  Y. Mansour,et al.  Algorithmic Game Theory: Learning, Regret Minimization, and Equilibria , 2007 .

[3]  Leslie G. Valiant,et al.  Evolvability , 2009, JACM.

[4]  Peter Secretan Learning , 1965, Mental Health.

[5]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[6]  Manfred K. Warmuth,et al.  The Weighted Majority Algorithm , 1994, Inf. Comput..

[7]  David E. Goldberg,et al.  Genetic algorithms and Machine Learning , 1988, Machine Learning.

[8]  Tilman Börgers,et al.  Learning Through Reinforcement and Replicator Dynamics , 1997 .

[9]  C. Smith,et al.  An Inequality Arising in Genetical Theory , 1959 .

[10]  Y. Freund,et al.  Adaptive game playing using multiplicative weights , 1999 .

[11]  T. Nagylaki The evolution of multilocus systems under weak selection. , 1993, Genetics.

[12]  A Hastings,et al.  Stable cycling in discrete-time genetic models. , 1981, Proceedings of the National Academy of Sciences of the United States of America.

[13]  Satyen Kale Efficient algorithms using the multiplicative weights update method , 2007 .

[14]  Michael Wooldridge,et al.  Game Theory and Evolution , 2013, IEEE Intelligent Systems.

[15]  Ethan Akin Hopf bifurcation in the two locus genetic model , 1983 .

[16]  E. Hopkins Learning, Matching and Aggregation , 1995 .

[17]  M Slatkin,et al.  Interaction of selection and recombination in the fixation of negative-epistatic genes. , 1996, Genetical research.

[18]  Josef Hofbauer,et al.  A Hopf bifurcation theorem for difference equations approximating a differential equation , 1984 .

[19]  P. Gács,et al.  Algorithms , 1992 .

[20]  Yishay Mansour,et al.  On the convergence of regret minimization dynamics in concave games , 2009, STOC '09.

[21]  B. Charlesworth,et al.  Why sex and recombination? , 1998, Science.

[22]  Éva Tardos,et al.  Multiplicative updates outperform generic no-regret learning in congestion games: extended abstract , 2009, STOC '09.

[23]  Umesh Vazirani,et al.  Algorithms, games, and evolution , 2014, Proceedings of the National Academy of Sciences.

[24]  Christos H. Papadimitriou,et al.  Multiplicative updates in coordination games and the theory of evolution , 2012, ITCS '13.

[25]  Gunes Ercal,et al.  On No-Regret Learning, Fictitious Play, and Nash Equilibrium , 2001, ICML.

[26]  R. Punnett,et al.  The Genetical Theory of Natural Selection , 1930, Nature.

[27]  Elizabeth Sklar,et al.  Auctions, Evolution, and Multi-agent Learning , 2007, Adaptive Agents and Multi-Agents Systems.

[28]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[29]  Jason R. Marden,et al.  Payoff-Based Dynamics for Multiplayer Weakly Acyclic Games , 2009, SIAM J. Control. Optim..

[30]  R. Bürger Some Mathematical Models in Evolutionary Genetics , 2011 .

[31]  J. Hofbauer,et al.  Convergence of multilocus systems under weak epistasis or weak selection , 1999, Journal of mathematical biology.

[32]  David Haussler,et al.  How to use expert advice , 1993, STOC.

[33]  R. Lewontin ‘The Selfish Gene’ , 1977, Nature.

[34]  Yishay Mansour,et al.  Improved second-order bounds for prediction with expert advice , 2006, Machine Learning.

[35]  Marek Kisiel-Dorohinicki,et al.  The Application of Evolution Process in Multi-Agent World to the Prediction System , 1996 .