Incorporating Bayesian learning in agent-based simulation of stakeholders' negotiation

abstract This paper describes the incorporation of a Bayesian learning algorithm into an agent-based modeldesigned to simulate stakeholders’ negotiation when evaluating scenarios of land development. Theobjective is to facilitate reaching an agreement at an earlier stage in the negotiation by providing theopportunity to the proposer agent to learn his opponents’ preferences. The modeling approach is testedin the Elbow River watershed, in southern Alberta, Canada, that is under considerable pressure for landdevelopment due to the proximity of the fast growing city of Calgary. Five agents are included in themodel respectively referred to as the Developer agent, the Planner agent, the Citizen agent, the Agriculture-Concerned agent, and the WaterConcerned agent. Two types of land development scenarios are evaluated;in the first case, only the geographical location is considered while in the second case, the internal land-use composition is also varied. The Developer agent that is equipped with the Bayesian learning capabilityattempts to approximate its opponents’ fuzzy evaluation functions based on the responses he receivesfrom them at each round of the negotiation. The results indicate that using this approach, an agreementcan be reached in fewer number of negotiation rounds than in the case where the Developer agent selectsthe subsequent offers based merely on its own utility. The model also indicates how the satisfaction ofeach agent evolves during the negotiation. This information is very useful for decision makers who wishto consider stakeholders’ perspectives when dealing with multiple objectives in a spatial context. 2014 Elsevier Ltd. All rights reserved.

[1]  Fulong Wu A linguistic cellular automata simulation approach for sustainable land development in a fast growing region , 1996 .

[2]  Shouhong Wang Generating fuzzy membership functions: a monotonic neural network model , 1994 .

[3]  Alberta. Guidelines for the approval and design of natural and constructed treatment wetlands for water quality improvement. , 1998 .

[4]  Stephen J. Walsh,et al.  GIS implications for hydrologic modeling: Simulation of nonpoint pollution generated as a consequence of watershed development scenarios , 1992 .

[5]  Isabelle Bichindaritz,et al.  Report on the Eighteenth International Conference on Case-Based Reasoning , 2012, AI Mag..

[6]  Sotiris B. Kotsiantis,et al.  Supervised Machine Learning: A Review of Classification Techniques , 2007, Informatica.

[7]  Suzana Dragicevic,et al.  Modeling-in-the-middle: bridging the gap between agent-based modeling and multi-objective decision-making for land use change , 2011, Int. J. Geogr. Inf. Sci..

[8]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[9]  Michael Batty,et al.  Modelling Inside GIS: Part 1. Model Structures, Exploratory Spatial Data Analysis and Aggregation , 1994, Int. J. Geogr. Inf. Sci..

[10]  Csaba Szepesvári,et al.  Algorithms for Reinforcement Learning , 2010, Synthesis Lectures on Artificial Intelligence and Machine Learning.

[11]  Ethem Alpaydin,et al.  Introduction to machine learning , 2004, Adaptive computation and machine learning.

[12]  I. Turksen,et al.  Measurement of Membership Functions: Theoretical and Empirical Work , 2000 .

[13]  Virginia Dignum,et al.  Towards Agent-Based Scenario Development for Strategic Decision Support , 2006, AOIS.

[14]  Kai N. Lee Compass and Gyroscope: Integrating Science and Politics for the Environment, Kai N. Lee. 1993. Island Press, Washington, DC. 290 pages. ISBN: 1-59963-197-X. $25.00 , 1993 .

[15]  Svetha Venkatesh,et al.  Learning Other Agents' Preferences in Multi-Agent Negotiation Using the Bayesian Classifier , 1999, Int. J. Cooperative Inf. Syst..

[16]  Mu-Song Chen,et al.  Fuzzy clustering analysis for optimizing fuzzy membership functions , 1999, Fuzzy Sets Syst..

[17]  Arie Segev,et al.  Automated Negotiations: A Survey of the State of the Art , 1997, Wirtschaftsinf..

[18]  Richard S. Sutton,et al.  Generalization in ReinforcementLearning : Successful Examples UsingSparse Coarse , 1996 .

[19]  Aaron M. Ellison,et al.  AN INTRODUCTION TO BAYESIAN INFERENCE FOR ECOLOGICAL RESEARCH AND ENVIRONMENTAL , 1996 .

[20]  Michael Batty,et al.  GIS and remote sensing as tools for the simulation of urban land‐use change , 2005 .

[21]  Piotr Jankowski,et al.  Exploring normative scenarios of land use development decisions with an agent-based simulation laboratory , 2010, Comput. Environ. Urban Syst..

[22]  S. Daniels,et al.  Collaborative learning: Improving public deliberation in ecosystem-based management , 1996 .

[23]  Nicholas R. Jennings,et al.  Using similarity criteria to make negotiation trade-offs , 2000, Proceedings Fourth International Conference on MultiAgent Systems.

[24]  Iyad Rahwan,et al.  Intelligent Agents for Automated One-to-Many E-Commerce Negotiation , 2002, ACSC.

[25]  Ruud Kempener,et al.  A complex systems approach to planning, optimization and decision making for energy networks , 2008 .

[26]  Piet Rietveld,et al.  LAND USE SCANNER: An integrated GIS based model for long term projections of land use in urban and rural areas , 1999, J. Geogr. Syst..

[27]  Douglas Gale,et al.  Bayesian learning in social networks , 2003, Games Econ. Behav..

[28]  N. R. Jennings,et al.  To appear in: Int Journal of Group Decision and Negotiation GDN2000 Keynote Paper Automated Negotiation: Prospects, Methods and Challenges , 2022 .

[29]  Robert Lempert,et al.  Agent-based modeling as organizational and public policy simulators , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[30]  J. Gareth Polhill,et al.  Agent-based land-use models: a review of applications , 2007, Landscape Ecology.

[31]  I. Turksen,et al.  Elicitation of membership functions: how far can theory take us? , 1997, Proceedings of 6th International Fuzzy Systems Conference.

[32]  Suzana Dragicevic,et al.  Simulation and validation of a reinforcement learning agent-based model for multi-stakeholder forest management , 2010, Comput. Environ. Urban Syst..

[33]  Majeed Pooyandeh,et al.  A spatial web/agent-based model to support stakeholders' negotiation regarding land development. , 2013, Journal of environmental management.

[34]  Parag Kulkarni Reinforcement and Systemic Machine Learning for Decision Making , 2012 .

[35]  Gerhard Weiß,et al.  Adaptation and Learning in Multi-Agent Systems: Some Remarks and a Bibliography , 1995, Adaption and Learning in Multi-Agent Systems.

[36]  Jean-Lou Chameau,et al.  Membership functions I: Comparing methods of measurement , 1987, Int. J. Approx. Reason..

[37]  Andrew W. Moore,et al.  Generalization in Reinforcement Learning: Safely Approximating the Value Function , 1994, NIPS.

[38]  Keith C. Clarke,et al.  The Use of Scenarios in Land-Use Planning , 2003 .

[39]  I. Turksen Measurement of membership functions and their acquisition , 1991 .

[40]  Bruce Spencer,et al.  A Bayesian classifier for learning opponents' preferences in multi-object automated negotiation , 2007, Electron. Commer. Res. Appl..

[41]  Lawrence M. Ausubel,et al.  Bargaining in Incomplete Information , 2002 .

[42]  Jian Li,et al.  Bayesian learning in bilateral multi-issue negotiation and its application in MAS-based electronic commerce , 2004 .

[43]  Jim R. Oliver A Machine-Learning Approach to Automated Negotiation and Prospects for Electronic Commerce , 1996, J. Manag. Inf. Syst..

[44]  Pedro M. Domingos,et al.  On the Optimality of the Simple Bayesian Classifier under Zero-One Loss , 1997, Machine Learning.

[45]  Michael Wooldridge,et al.  Negotiation among autonomous computational agents: principles, analysis and challenges , 2008, Artificial Intelligence Review.

[46]  John Forester,et al.  The Deliberative Practitioner: Encouraging Participatory Planning Processes , 1999 .

[47]  Enrico H. Gerding,et al.  Scientific approaches and techniques for negotiation. A game theoretic and artificial intelligence perspective , 2000 .

[48]  Meine van Noordwijk,et al.  Negotiation Support Models for Integrated Natural Resource Management in Tropical Forest Margins , 2002 .

[49]  Robert A Jacobs,et al.  Bayesian learning theory applied to human cognition. , 2011, Wiley interdisciplinary reviews. Cognitive science.

[50]  Katia P. Sycara,et al.  Bayesian learning in negotiation , 1998, Int. J. Hum. Comput. Stud..

[51]  Gregory E. Kersten,et al.  Predicting opponent's moves in electronic negotiations using neural networks , 2008, Expert Syst. Appl..

[52]  D. Marceau,et al.  Simulating a Land Development Planning Process through Agent-Based Modeling , 2011 .

[53]  Koen V. Hindriks,et al.  Opponent modelling in automated multi-issue negotiation using Bayesian learning , 2008, AAMAS.

[54]  Jiming Liu,et al.  A genetic agent-based negotiation system , 2001, Comput. Networks.

[55]  M. Bazerman Judgment in Managerial Decision Making , 1990 .

[56]  Michael Winikoff,et al.  Agent-oriented Information Systems IV , 2008 .

[57]  Chimay J. Anumba,et al.  Learning in multi-agent systems: a case study of construction claims negotiation , 2002, Adv. Eng. Informatics.

[58]  Sean Luke,et al.  Cooperative Multi-Agent Learning: The State of the Art , 2005, Autonomous Agents and Multi-Agent Systems.