Applying the dynamics of evolution to achieve reliability in master–worker computing

We consider Internet‐based master–worker task computations, such as SETI@home, where a master process sends tasks, across the Internet, to worker processes; workers execute and report back some result. However, these workers are not trustworthy, and it might be at their best interest to report incorrect results. In such master–worker computations, the behavior and the best interest of the workers might change over time. We model such computations using evolutionary dynamics, and we study the conditions under which the master can reliably obtain task results. In particular, we develop and analyze an algorithmic mechanism based on reinforcement learning to provide workers with the necessary incentives to eventually become truthful. Our analysis identifies the conditions under which truthful behavior can be ensured and bounds the expected convergence time to that behavior. The analysis is complemented with illustrative simulations. Copyright © 2013 John Wiley & Sons, Ltd.

[1]  Moshe Babaioff,et al.  Combinatorial agency , 2006, EC '06.

[2]  Moshe Babaioff,et al.  Mixed Strategies in Combinatorial Agency , 2006, WINE.

[3]  Sanguthevar Rajasekaran,et al.  Robust Network Supercomputing with Malicious Processes , 2006, DISC.

[4]  Chryssis Georgiou,et al.  Algorithmic mechanisms for internet-based master-worker computing with untrusted and selfish workers , 2010, 2010 IEEE International Symposium on Parallel & Distributed Processing (IPDPS).

[5]  David P. Anderson,et al.  BOINC: a system for public-resource computing and storage , 2004, Fifth IEEE/ACM International Workshop on Grid Computing.

[6]  Jörgen W. Weibull,et al.  Evolutionary Game Theory , 1996 .

[7]  Tilman Börgers,et al.  Learning Through Reinforcement and Replicator Dynamics , 1997 .

[8]  Stefan Schmid,et al.  Combinatorial agency with audits , 2009, 2009 International Conference on Game Theory for Networks.

[9]  J. M. Smith,et al.  The Logic of Animal Conflict , 1973, Nature.

[10]  Michael D. Buhrmester,et al.  Amazon's Mechanical Turk , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[11]  Elias Koutsoupias,et al.  Mechanism Design for Scheduling , 2009, Bull. EATCS.

[12]  Jonathan Bendor,et al.  Aspiration-Based Reinforcement Learning in Repeated Games: An Overview , 2001 .

[13]  Moshe Babaioff,et al.  Free-Riding and Free-Labor in Combinatorial Agency , 2009, SAGT.

[14]  Vincent Conitzer,et al.  Incremental Mechanism Design , 2007, IJCAI.

[15]  Chryssis Georgiou,et al.  Designing Mechanisms for Reliable Internet-based Computing , 2008, 2008 Seventh IEEE International Symposium on Network Computing and Applications.

[16]  R. Axelrod,et al.  Evolutionary Dynamics , 2004 .

[17]  David P. Anderson,et al.  Performance Prediction and Analysis of BOINC Projects: An Empirical Study with EmBOINC , 2009, Journal of Grid Computing.

[18]  E. David,et al.  Networks, Crowds, and Markets: Reasoning about a Highly Connected World , 2010 .

[19]  Dirk Ifenthaler,et al.  Stochastic Models of Learning , 2012 .

[20]  Luis F. G. Sarmenta.  Sabotage-tolerance mechanisms for volunteer computing systems , 2002, Future Gener. Comput. Syst..

[21]  David P. Anderson,et al.  Homogeneous redundancy: a technique to ensure integrity of molecular simulation results using public computing , 2005, 19th IEEE International Parallel and Distributed Processing Symposium.

[22]  David C. Parkes,et al.  Rationality and Self-Interest in Peer to Peer Networks , 2003, IPTPS.

[23]  Colin Camerer.  Behavioral Game Theory: Experiments in Strategic Interaction , 2003 .

[24]  L. Samuelson.  Evolutionary Games and Equilibrium Selection , 1997 .

[25]  Joseph Y. Halpern.  Computer Science and Game Theory: A Brief Survey , 2007, ArXiv.

[26]  Peter McBurney,et al.  Autonomous Agents and Multi-agent Systems Manuscript No. Evolutionary Mechanism Design: a Review , 2022 .

[27]  Ron Lavi,et al.  Algorithmic Mechanism Design , 2008, Encyclopedia of Algorithms.

[28]  Jean-François Laslier,et al.  A Behavioral Learning Process in Games , 2001, Games Econ. Behav..

[29]  J. Nash.  Equilibrium Points in N-Person Games. , 1950, Proceedings of the National Academy of Sciences of the United States of America.

[30]  Chris Arney,et al.  Networks, Crowds, and Markets: Reasoning about a Highly Connected World (Easley, D. and Kleinberg, J.; 2010) [Book Review] , 2013, IEEE Technology and Society Magazine.

[31]  J. Cross.  A theory of adaptive economic behavior , 1983 .

[32]  D. Fudenberg,et al.  The Theory of Learning in Games , 1998 .

[33]  J. Henrich,et al.  The evolution of cultural evolution , 2003 .

[34]  David P. Anderson,et al.  SETI@home: an experiment in public-resource computing , 2002, CACM.

[35]  Jonathan Bendor,et al.  Reinforcement Learning in Repeated Interaction Games , 2001 .

[36]  Ittai Abraham,et al.  Distributed computing meets game theory: combining insights from two fields , 2011, SIGA.

[37]  David Rose,et al.  The principal-agent problem with evolutionary learning , 1996, Comput. Math. Organ. Theory.

[38]  Philippe Golle,et al.  Uncheatable Distributed Computations , 2001, CT-RSA.

[39]  Csaba Szepesvári,et al.  Algorithms for Reinforcement Learning , 2010, Synthesis Lectures on Artificial Intelligence and Machine Learning.

[40]  Segismundo S. Izquierdo,et al.  Dynamics of the Bush-Mosteller Learning Algorithm in 2x2 Games , 2008 .

[41]  Chryssis Georgiou,et al.  Reliably Executing Tasks in the Presence of Untrusted Entities , 2006, 2006 25th IEEE Symposium on Reliable Distributed Systems (SRDS'06).

[42]  Frederick Mosteller,et al.  Stochastic Models for Learning , 1956 .

[43]  J M Smith,et al.  Evolution and the theory of games , 1976 .

[44]  Danny Dolev,et al.  Distributed computing meets game theory: robust mechanisms for rational secret sharing and multiparty computation , 2006, PODC '06.

[45]  J. Weibull,et al.  Nash Equilibrium and Evolution by Imitation , 1994 .

[46]  Kenichi Hagihara,et al.  Computing Low Latency Batches with Unreliable Workers in Volunteer Computing Environments , 2008, 2008 IEEE International Symposium on Parallel and Distributed Processing.

[47]  Jonathan Bendor,et al.  Aspiration-Based Reinforcement Learning in Repeated Interaction Games: an Overview , 2001, IGTR.

[48]  Tim Roughgarden,et al.  Algorithmic Game Theory , 2007 .

[49]  David P. Anderson,et al.  SETI@home-massively distributed computing for SETI , 2001, Comput. Sci. Eng..

[50]  Arnold L. Rosenberg,et al.  On the cost-ineffectiveness of redundancy in commercial P2P computing , 2005, CCS '05.