Self-Organized Cooperation Policy Setting in P2P Systems Based on Reinforcement Learning

In this paper, we have developed a self-organized approach to cooperation policy setting in a system of rational peers that have only partial views of the whole system in order to improve the overall welfare as a system-wide performance metric. The proposed approach is based on distributed reinforcement learning and sets cooperation policies of the peers through their self-organized interactions. We have analyzed this approach to demonstrate that it results in Pareto optimality in the system by disseminating the local value functions of the peers among the neighbors. We have also experimentally verified that this approach outperforms the other commonly used approaches in the literature, in terms of the performance of the system.

[1]  Yong Liu,et al.  Incentivized Peer-Assisted Streaming for On-Demand Services , 2010, IEEE Transactions on Parallel and Distributed Systems.

[2]  John C.-I. Chuang,et al.  Service differentiated peer selection: an incentive mechanism for peer-to-peer media streaming , 2006, IEEE Transactions on Multimedia.

[3]  Panayotis Antoniadis,et al.  Comparing economic incentives in peer-to-peer networks , 2004, Comput. Networks.

[4]  Yeh-Ching Chung,et al.  Direction-aware resource discovery service in large-scale grid and cloud computing , 2011, 2011 IEEE International Conference on Service-Oriented Computing and Applications (SOCA).

[5]  Ion Stoica,et al.  Robust incentive techniques for peer-to-peer networks , 2004, EC '04.

[6]  Neil Immerman,et al.  The Complexity of Decentralized Control of Markov Decision Processes , 2000, UAI.

[7]  Sean Luke,et al.  Cooperative Multi-Agent Learning: The State of the Art , 2005, Autonomous Agents and Multi-Agent Systems.

[8]  Jacques L. Koko,et al.  The Art and Science of Negotiation , 2009 .

[9]  Panayotis Antoniadis,et al.  Incentives for content availability in memory-less peer-to-peer file sharing systems , 2005, SECO.

[10]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[11]  Siavash Khorsandi,et al.  Coordination of cooperation policies in a peer-to-peer system using swarm-based RL , 2012, J. Netw. Comput. Appl..

[12]  Haibin Lu,et al.  Incentive Schemes in Peer-to-Peer Networks , 2008 .

[13]  Kavitha Ranganathan,et al.  Incentive mechanisms for large collaborative resource sharing , 2004, IEEE International Symposium on Cluster Computing and the Grid, 2004. CCGrid 2004..

[14]  Stuart M. Allen,et al.  Cooperation through self-similar social networks , 2010, TAAS.

[15]  S. Khaddaj,et al.  A Brokerage Framework for Intelligent Resource Allocation in Distributed Systems , 2010, 2010 Ninth International Symposium on Distributed Computing and Applications to Business, Engineering and Science.

[16]  Mihaela van der Schaar,et al.  Coalition-Based Resource Reciprocation Strategies for P2P Multimedia Broadcasting , 2008, IEEE Transactions on Broadcasting.

[17]  Stephen A. Jarvis,et al.  A Payment-Based Incentive and Service Differentiation Scheme for Peer-to-Peer Streaming Broadcast , 2008, IEEE Transactions on Parallel and Distributed Systems.

[18]  Thinh P. Nguyen,et al.  A Global Contribution Approach to Maintain Fairness in P2P Networks , 2010, IEEE Transactions on Parallel and Distributed Systems.

[19]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[20]  Francis Heylighen,et al.  The Science of Self-Organization and Adaptivity , 1999 .

[21]  Divyakant Agrawal,et al.  A game theoretic framework for incentives in P2P systems , 2003, Proceedings Third International Conference on Peer-to-Peer Computing (P2P2003).

[22]  Mihaela van der Schaar,et al.  Stochastic Optimization for Content Sharing in P2P Systems , 2008, IEEE Transactions on Multimedia.

[23]  Leandros Tassiulas,et al.  Reputation-Based Resource Allocation in P2P Systems of Rational Users , 2010, IEEE Transactions on Parallel and Distributed Systems.

[24]  Ting Li,et al.  PIRD: P2P-Based Intelligent Resource Discovery in Internet-Based Distributed Systems , 2008, 2008 The 28th International Conference on Distributed Computing Systems.

[25]  Bart De Schutter,et al.  A Comprehensive Survey of Multiagent Reinforcement Learning , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[26]  Roger B. Myerson,et al.  Game theory - Analysis of Conflict , 1991 .

[27]  David K. Y. Yau,et al.  Incentive and Service Differentiation in P2P Networks: A Game Theoretic Approach , 2006, IEEE/ACM Transactions on Networking.