Using temporal-difference learning for multi-agent bargaining

This research treats a bargaining process as a Markov decision process, in which a bargaining agent's goal is to learn the optimal policy that maximizes the total rewards it receives over the process. Reinforcement learning is an effective method for agents to learn how to determine actions for any time steps in a Markov decision process. Temporal-difference (TD) learning is a fundamental method for solving the reinforcement learning problem, and it can tackle the temporal credit assignment problem. This research designs agents that apply TD-based reinforcement learning to deal with online bilateral bargaining with incomplete information. This research further evaluates the agents' bargaining performance in terms of the average payoff and settlement rate. The results show that agents using TD-based reinforcement learning are able to achieve good bargaining performance. This learning approach is sufficiently robust and convenient, hence it is suitable for online automated bargaining in electronic commerce.

[1]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[2]  Katia P. Sycara,et al.  A computational model for online agent negotiation , 2002, Proceedings of the 35th Annual Hawaii International Conference on System Sciences.

[3]  Maureen Caudill,et al.  Neural networks primer, part III , 1988 .

[4]  Steven Guan,et al.  A factory-based approach to support e-commerce agent fabrication , 2004, Electron. Commer. Res. Appl..

[5]  Katia P. Sycara,et al.  Bayesian learning in negotiation , 1998, Int. J. Hum. Comput. Stud..

[6]  Stan Matwin,et al.  Genetic algorithms approach to a negotiation support system , 1991, IEEE Trans. Syst. Man Cybern..

[7]  Paolo Torroni,et al.  Dialogues for Negotiation: Agent Varieties and Dialogue Sequences , 2001, ATAL.

[8]  Long Ji Lin,et al.  Self-improving reactive agents based on reinforcement learning, planning and teaching , 1992, Machine Learning.

[9]  Gerald Tesauro,et al.  Temporal difference learning and TD-Gammon , 1995, CACM.

[10]  Jim R. Oliver.  A Machine-Learning Approach to Automated Negotiation and Prospects for Electronic Commerce , 1996, J. Manag. Inf. Syst..

[11]  Guido Governatori,et al.  A formal approach to negotiating agents development , 2002, Electron. Commer. Res. Appl..

[12]  Fillia Makedon,et al.  A hybrid negotiation strategy mechanism in an automated negotiation system , 2004, EC '04.

[13]  Chris Watkins,et al.  Learning from delayed rewards , 1989 .

[14]  Steven O. Kimbrough,et al.  Bargaining by artificial agents in two coalition games: a study in genetic programming for electronic commerce , 1996 .

[15]  Garett O. Dworman,et al.  On automated discovery of models using genetic programming in game-theoretic contexts , 1995, Proceedings of the Twenty-Eighth Annual Hawaii International Conference on System Sciences.

[16]  P. Dasgupta,et al.  The Economics of Bargaining , 1990 .

[17]  Bruce Spencer,et al.  NRC Publications Archive Archives des publications du CNRC A Bayesian classifier for learning opponents' preferences in multi-object automated negotiation , 2007 .

[18]  Robert T. Clemen,et al.  Making Hard Decisions with Decisiontools Suite , 2000 .

[19]  Sarit Kraus,et al.  Reaching Agreements Through Argumentation: A Logical Model and Implementation , 1998, Artif. Intell..

[20]  E. Mine Cinar,et al.  Neural Networks: A New Tool for Predicting Thrift Failures , 1992 .

[21]  Fu-Ren Lin,et al.  A Multiagent Framework for Automated Online Bargaining , 2001, IEEE Intell. Syst..

[22]  Felix A. Fischer,et al.  Cooperative Information Agents XI , 2008 .

[23]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[24]  Julita Vassileva,et al.  Bilateral Negotiation with Incomplete and Uncertain Information: A Decision-Theoretic Approach Using a Model of the Opponent , 2000, CIA.

[25]  Ayman M. Wasfy,et al.  Two-Party Negotiation Modeling: An Integrated Fuzzy Logic Approach , 1998 .

[26]  Leen-Kiat Soh,et al.  Agent-Based Argumentative Negotiations with Case-Based Reasoning , 2001 .

[27]  Jianhua Ma,et al.  An experience-based evolutionary negotiation model , 2003, Proceedings Fifth International Conference on Computational Intelligence and Multimedia Applications. ICCIMA 2003.