Distributed Constrained Optimization with Semicoordinate Transformations

Recent work has shown how information theory extends conventional full-rationality game theory to allow bounded rational agents. The associated mathematical framework can be used to solve constrained optimization problems. This is done by translating the problem into an iterated game, where each agent controls a different variable of the problem, so that the joint probability distribution across the agents moves gives an expected value of the objective function. The dynamics of the agents is designed to minimize a Lagrangian function of that joint distribution. Here we illustrate how the updating of the Lagrange parameters in the Lagrangian is a form of automated annealing, which focuses the joint distribution more and more tightly about the joint moves that optimize the objective function. We then investigate the use of "semicoordinate" variable transformations. These separate the joint state of the agents from the variables of the optimization problem, with the two connected by an onto mapping. We present experiments illustrating the ability of such transformations to facilitate optimization. We focus on the special kind of transformation in which the statistically independent states of the agents induces a mixture distribution over the optimization variables. Computer experiment illustrate this for &sat constraint satisfaction problems and for unconstrained minimization of NK functions.

[1]  E. Jaynes.  Information Theory and Statistical Mechanics , 1957 .

[2]  David G. Stork,et al.  Pattern Classification , 1973 .

[3]  T. Başar,et al.  Dynamic Noncooperative Game Theory , 1982 .

[4]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[5]  R. Aumann.  Correlated Equilibrium as an Expression of Bayesian Rationality Author ( s ) , 1987 .

[6]  S. Kauffman,et al.  Towards a general theory of adaptive walks on rugged landscapes. , 1987, Journal of theoretical biology.

[7]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[8]  Jianhua Lin,et al.  Divergence measures based on the Shannon entropy , 1991, IEEE Trans. Inf. Theory.

[9]  S. Hart,et al.  Handbook of Game Theory with Economic Applications , 1992 .

[10]  Liam Murphy,et al.  Distributed Constrained Optimization , 1993, SIAM Conference on Parallel Processing for Scientific Computing.

[11]  Bart Selman,et al.  Local search strategies for satisfiability testing , 1993, Cliques, Coloring, and Satisfiability.

[12]  Ariel Rubinstein,et al.  A Course in Game Theory , 1995 .

[13]  Michael I. Jordan,et al.  Reinforcement Learning by Probability Matching , 1995, NIPS 1995.

[14]  Paul A. Viola,et al.  MIMIC: Finding Optima by Estimating Probability Densities , 1996, NIPS.

[15]  M. Trick,et al.  Cliques, Coloring, and Satisfiability: Second DIMACS Implementation Challenge, Workshop, October 11-13, 1993 , 1996 .

[16]  Saul I. Gass,et al.  Encyclopedia of Operations Research and Management Science , 1997 .

[17]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[18]  Michael I. Jordan,et al.  Improving the Mean Field Approximation Via the Use of Mixture Distributions , 1999, Learning in Graphical Models.

[19]  Kagan Tumer,et al.  Using Collective Intelligence to Route Internet Traffic , 1998, NIPS.

[20]  D. Fudenberg,et al.  The Theory of Learning in Games , 1998 .

[21]  Rajeev Sharma,et al.  Advances in Neural Information Processing Systems 11 , 1999 .

[22]  Kagan Tumer,et al.  Collective Intelligence for Control of Distributed Dynamical Systems , 1999, ArXiv.

[23]  Peter Sollich,et al.  Advances in neural information processing systems 11 , 1999 .

[24]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[25]  J. Pollack,et al.  A game-theoretic investigation of selection methods used in evolutionary algorithms , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[26]  G. Theraulaz,et al.  Inspiration for optimization from social insect behaviour , 2000, Nature.

[27]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[28]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[29]  Kagan Tumer,et al.  Optimal Payoff Functions for Members of Collectives , 2001, Adv. Complex Syst..

[30]  Shumeet Baluja,et al.  Using a priori knowledge to create probabilistic models for optimization , 2002, Int. J. Approx. Reason..

[31]  Kagan Tumer,et al.  Collective Intelligence, Data Routing and Braess' Paradox , 2002, J. Artif. Intell. Res..

[32]  M. Mézard,et al.  Analytic and Algorithmic Solution of Random Satisfiability Problems , 2002, Science.

[33]  R. Durrett,et al.  RIGOROUS RESULTS FOR THE NK MODEL , 2003 .

[34]  R. Durrett,et al.  Rigorous results for the N K model , 2003 .

[35]  David H. Wolpert,et al.  Information Theory - The Bridge Connecting Bounded Rational Game Theory and Statistical Physics , 2004, ArXiv.

[36]  Kagan Tumer,et al.  Improving Search Algorithms by Using Intelligent Coordinates , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[37]  D. Wolpert.  Finding Bounded Rational Equilibria. Part 1; Iterative Focusing , 2004 .

[38]  Kagan Tumer,et al.  Collectives and Design Complex Systems , 2004 .

[39]  D. Wolpert.  Finding Bounded Rational Equilibria. Part 2; Alternative Lagrangians and Uncountable Move Spaces , 2004 .

[40]  David H. Wolpert,et al.  Product distribution theory for control of multi-agent systems , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[41]  David H. Wolpert,et al.  Discrete, Continuous, and Constrained Optimization Using Collectives , 2004 .

[42]  Dirk P. Kroese,et al.  Combinatorial Optimization via Cross-Entropy , 2004 .

[43]  Dirk P. Kroese,et al.  Cross‐Entropy Method , 2011 .

[44]  David H. Wolpert,et al.  Adaptive, distributed control of constrained multi-agent systems , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[45]  Ilan Kroo,et al.  Fleet Assignment Using Collective Intelligence , 2004 .

[46]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[47]  David H. Wolpert,et al.  Distributed control by Lagrangian steepest descent , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[48]  Jeff S. Shamma,et al.  Dynamic fictitious play, dynamic gradient play, and distributed convergence to Nash equilibria , 2005, IEEE Transactions on Automatic Control.

[49]  R. Rubinstein.  A Stochastic Minimum Cross-Entropy Method for Combinatorial Optimization and Rare-event Estimation* , 2005 .

[50]  Sevan G. Ficici.  A game-theoretic investigation of selection methods in two-population coevolution , 2006, GECCO '06.

[51]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[52]  David H. Wolpert,et al.  Advances in Distributed Optimization Using Probability Collectives , 2006, Adv. Complex Syst..

[53]  Pedro Larrañaga,et al.  Towards a New Evolutionary Computation - Advances in the Estimation of Distribution Algorithms , 2006, Towards a New Evolutionary Computation.

[54]  W. Grantham.  Gradient Transformation Trajectory Following Algorithms for Determining Stationary Min-Max Saddle Points , 2007 .

[55]  Roman Barták,et al.  Constraint Processing , 2009, Encyclopedia of Artificial Intelligence.

[56]  W. Macready.  Distributed Constrained Optimization , .