Local optimization in cooperative agent networks

My research focuses on constructing and analyzing systems of intelligent, autonomous agents. These agents may include people, physical robots, or software programs acting as assistants, teammates, opponents, or trading partners. In a large class of multi-agent scenarios, the effect of local interactions between agents can be compactly represented as a network structure such as a distributed constraint optimization problem (DCOP) for cooperative domains. Collaboration between large groups of agents, given such a network, can be difficult to achieve; often agents can only manage to collaborate in smaller subgroups of a certain size, in order to find a workable solution in a timely manner. The goal of my thesis is to provide algorithms to enable networks of agents that are bounded in this way to quickly find high-quality solutions, as well as theoretical results to understand key properties of these solutions. Relevant domains for my work include personal assistant agents, sensor networks, and teams of autonomous robots. In particular, this thesis considers the case in which agents optimize a DCOP by forming groups of one or more agents until no group of k or fewer agents can possibly improve the solution; we define this type of local optimum, and any algorithm guaranteed to reach such a local optimum, as k-optimal. In this document, I present four key contributions related to k-optimality. The first set of results are worst-case guarantees on the solution quality of k-optima in a DCOP. These guarantees can help determine an appropriate k-optimal algorithm, or possibly an appropriate constraint graph structure, for agents to use in situations where the cost of coordination between agents must be weighed against the quality of the solution reached. The second set of results are upper bounds on the number of k-optima that can exist in a DCOP. Because each joint action consumes resources, knowing the maximal number of k-optimal joint actions that could exist for a given DCOP allows us to allocate sufficient resources for a given level of k, or, alternatively, choosing an appropriate level of k- optimality, given fixed resource. The third contribution is a set of 2-optimal and 3-optimal algorithms and an experimental analysis of the performance of 1-, 2-, and 3-optimal algorithms on several types of DCOPs. The final contribution of this thesis is a case study of the application of k-optimal DCOP algorithms and solutions to the problem of the formation of human teams spanning multiple organizations. Given a particular specification of a human team (such as a task force to respond to an emergency) and a pool of possible team members, a DCOP can be formulated to match this specification. A set of k-optimal solutions to the DCOP represents a set of diverse, locally optimal options from which a human commander can choose the team that will be used.

[1]  Milind Tambe,et al.  How local is that optimum? k-optimality for DCOP , 2005, AAMAS '05.

[2]  Milind Tambe,et al.  Conflicts in teamwork: hybrids to the rescue , 2005, AAMAS '05.

[3]  H. Keiding On the Maximal Number of Nash Equilibria in ann × nBimatrix Game , 1997 .

[4]  Daphne Koller,et al.  Multi-agent algorithms for solving graphical games , 2002, AAAI/IAAI.

[5]  Makoto Yokoo,et al.  Adopt: asynchronous distributed constraint optimization with quality guarantees , 2005, Artif. Intell..

[6]  Milind Tambe,et al.  DCOP Games for Multi-agent Coordination , 2005 .

[7]  John Levine,et al.  Generation of Multiple Qualitatively Different Plan Options , 1998, AIPS.

[8]  Milind Tambe,et al.  Experimental analysis of privacy loss in DCOP algorithms , 2006, AAMAS '06.

[9]  Milind Tambe,et al.  Multiagent Teamwork: Hybrid Approaches , 2006 .

[10]  Vincent Conitzer,et al.  Complexity Results about Nash Equilibria , 2002, IJCAI.

[11]  Michael L. Littman,et al.  An Efficient, Exact Algorithm for Solving Tree-Structured Graphical Games , 2001, NIPS.

[12]  Weixiong Zhang,et al.  Distributed stochastic search and distributed breakout: properties, comparison and applications to constraint optimization problems in sensor networks , 2005, Artif. Intell..

[13]  Makoto Yokoo,et al.  Networked Distributed POMDPs: A Synergy of Distributed Constraint Optimization and POMDPs , 2005, IJCAI.

[14]  Chaoping Xing,et al.  Coding Theory: A First Course , 2004 .

[15]  Victor Lesser,et al.  Negotiation among self-interested computationally limited agents , 1996 .

[16]  Weixiong Zhang,et al.  An analysis and application of distributed constraint satisfaction and optimization algorithms in sensor networks , 2003, AAMAS '03.

[17]  Stephen Fitzpatrick,et al.  Distributed Coordination through Anarchic Optimization , 2003 .

[18]  Gregory Gutin,et al.  Domination Analysis of Combinatorial Optimization Algorithms and Problems , 2005 .

[19]  Milind Tambe Electric Elves: What Went Wrong and Why , 2006, AI Mag..

[20]  John P. Lewis,et al.  The DEFACTO System: Training Tool for Incident Commanders , 2005, AAAI.

[21]  Milind Tambe,et al.  Valuations of Possible States (VPS): a quantitative framework for analysis of privacy loss among collaborative personal assistant agents , 2005, AAMAS '05.

[22]  Milind Tambe,et al.  A Family of Graphical-Game-Based Algorithms for Distributed Constraint Optimization Problems , 2006 .

[23]  Milind Tambe,et al.  Local Algorithms for Distributed Constraint Optimization in Dynamic , Anytime Environments , 2005 .

[24]  Makoto Yokoo Why Adding More Constraints Makes a Problem Easier for Hill-climbing Algorithms: Analyzing Landscapes of CSPs , 1997, CP.

[26]  Victor R. Lesser,et al.  Solving distributed constraint optimization problems using cooperative mediation , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[27]  Jonathan P. Pearce,et al.  Qualitative behavior prediction for simple mechanical systems , 2001 .

[28]  Makoto Yokoo,et al.  Multiply-constrained distributed constraint optimization , 2006, AAMAS '06.

[29]  A. McLennan,et al.  Generic 4 x 4 Two Person Games Have at Most 15 Nash Equilibria , 1999 .

[30]  Soraya B. Rana,et al.  Representation Issues in Neighborhood Search and Evolutionary Algorithms , 1998 .

[31]  M. Yokoo,et al.  Distributed Breakout Algorithm for Solving Distributed Constraint Satisfaction Problems , 1996 .

[32]  Michael L. Littman,et al.  Graphical Models for Game Theory , 2001, UAI.

[33]  Milind Tambe,et al.  Graph-Based Bounds on k-Optimal Joint-Action Sets for Multiple Agents , 2005 .

[34]  Kim Jong Tae,et al.  New efficient clique partitioning algorithms for register-transfer synthesis of data paths , 2002 .

[35]  Eugene C. Freuder Synthesizing constraint expressions , 1978, CACM.

[36]  Milind Tambe,et al.  Preprocessing techniques for accelerating the DCOP algorithm ADOPT , 2005, AAMAS '05.

[37]  Milind Tambe,et al.  Distributed Algorithms for DCOP: A Graphical-Game-Based Approach , 2004, PDCS.

[38]  Boi Faltings,et al.  ODPOP: An Algorithm for Open/Distributed Constraint Optimization , 2006, AAAI.

[39]  Milind Tambe,et al.  Quality Guarantees on k-Optimal Solutions for Distributed Constraint Optimization Problems , 2007, IJCAI.

[40]  Milind Tambe,et al.  Taking DCOP to the real world: efficient complete solutions for distributed multi-event scheduling , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[41]  Sarit Kraus,et al.  An Efficient Heuristic for Security against Multiple Adversaries in Stackelberg Games , 2007, AAAI Spring Symposium: Game Theoretic and Decision Theoretic Agents.

[42]  Boi Faltings,et al.  A Scalable Method for Multiagent Constraint Optimization , 2005, IJCAI.

[43]  Shen Lin Computer solutions of the traveling salesman problem , 1965 .

[44]  Edmund H. Durfee,et al.  A distributed framework for solving the Multiagent Plan Coordination Problem , 2005, AAMAS '05.

[45]  Nikos A. Vlassis,et al.  Anytime algorithms for multiagent decision making using coordination graphs , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[46]  Noga Alon,et al.  Approximating the independence number via theϑ-function , 1998, Math. Program..

[47]  Milind Tambe,et al.  Privacy Loss in Distributed Constraint Reasoning: A Quantitative Framework for Analysis and its Applications , 2006, Autonomous Agents and Multi-Agent Systems.

[48]  Milind Tambe,et al.  Solution sets for DCOPs and graphical games , 2006, AAMAS '06.

[49]  Milind Tambe,et al.  Solution Sets for DCOPs and Graphical Games: Metrics and Bounds , 2006, ISAIM.

[50]  Jonathan P. Pearce Locally Optimal Algorithms and Solutions for Distributed Constraint Optimization , 2006, AAAI.

[51]  Sarit Kraus,et al.  An efficient heuristic approach for security against multiple adversaries , 2007, AAMAS '07.

[52]  Sui Ruan,et al.  Patrolling in a Stochastic Environment , 2005 .

[53]  Daphne Koller,et al.  A Continuation Method for Nash Equilibria in Structured Games , 2003, IJCAI.

[54]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.