The Multiplicative Weights Update Method: a Meta-Algorithm and Applications

Algorithms in varied fields use the idea of maintaining a distribution over a certain set and use the multiplicative update rule to iteratively change these weights. Their analyses are usually very similar and rely on an exponential potential function. In this survey we present a simple meta-algorithm that unifies many of these disparate algorithms and derives them as simple instantiations of the meta-algorithm. We feel that since this meta-algorithm and its analysis are so simple, and its applications so broad, it should be a standard part of algorithms courses, like "divide and conquer."

[1]  J. Neumann,et al.  SOLUTIONS OF GAMES BY DIFFERENTIAL EQUATIONS , 1950 .

[2]  J. Robinson.  AN ITERATIVE METHOD OF SOLVING A GAME , 1951, Classics in Game Theory.

[3]  S. Golden.  LOWER BOUNDS FOR THE HELMHOLTZ FUNCTION , 1965 .

[4]  C. Thompson.  Inequality with Applications in Statistical Mechanics , 1965 .

[5]  Andrew Chi-Chih Yao,et al.  Theory and Applications of Trapdoor Functions (Extended Abstract) , 1982, FOCS.

[6]  P. Raghavan.  Probabilistic construction of deterministic algorithms: Approximating packing integer programs , 1986, 27th Annual Symposium on Foundations of Computer Science (sfcs 1986).

[7]  Prabhakar Raghavan,et al.  Probabilistic construction of deterministic algorithms: Approximating packing integer programs , 1986, 27th Annual Symposium on Foundations of Computer Science (sfcs 1986).

[8]  N. Littlestone.  Learning Quickly When Irrelevant Attributes Abound: A New Linear-Threshold Algorithm , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[9]  Prabhakar Raghavan,et al.  Randomized rounding: A technique for provably good algorithms and algorithmic proofs , 1985, Comb..

[10]  Kenneth L. Clarkson,et al.  A Las Vegas algorithm for linear programming when the dimension is small , 1988, [Proceedings 1988] 29th Annual Symposium on Foundations of Computer Science.

[11]  Bernard Chazelle,et al.  Quasi-optimal range searching in spaces of finite VC-dimension , 1989, Discret. Comput. Geom..

[12]  N. Littlestone.  Mistake bounds and logarithmic linear-threshold learning algorithms , 1990 .

[13]  Éva Tardos,et al.  Fast approximation algorithms for fractional packing and covering problems , 1991, [1991] Proceedings 32nd Annual Symposium of Foundations of Computer Science.

[14]  Noga Alon,et al.  The Probabilistic Method , 2015, Fundamentals of Ramsey Theory.

[15]  Noam Nisan,et al.  A parallel approximation algorithm for positive linear programming , 1993, STOC.

[16]  Manfred K. Warmuth,et al.  The Weighted Majority Algorithm , 1994, Inf. Comput..

[17]  Baruch Awerbuch,et al.  Improved approximation algorithms for the multi-commodity flow problem and local competitive routing in dynamic networks , 1994, STOC '94.

[18]  David J. Goodman,et al.  Personal Communications , 1994, Mobile Communications.

[19]  Michael T. Goodrich,et al.  Almost optimal set covers in finite VC-dimension: (preliminary version) , 1994, SCG '94.

[20]  Neal E. Young,et al.  Randomized rounding without solving the linear program , 1995, SODA '95.

[21]  Leonid Khachiyan,et al.  A sublinear-time randomized approximation algorithm for matrix games , 1995, Oper. Res. Lett..

[22]  Russell Impagliazzo,et al.  Hard-core distributions for somewhat hard problems , 1995, Proceedings of IEEE 36th Annual Foundations of Computer Science.

[23]  Michael T. Goodrich,et al.  Almost optimal set covers in finite VC-dimension , 1995, Discret. Comput. Geom..

[24]  Kenneth L. Clarkson,et al.  Las Vegas algorithms for linear and integer programming when the dimension is small , 1995, JACM.

[25]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[26]  David P. Williamson,et al.  Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming , 1995, JACM.

[27]  David R. Karger,et al.  Approximating s-t minimum cuts in Õ(n2) time , 1996, STOC '96.

[28]  Hsueh-I Lu,et al.  Efficient approximation algorithms for semidefinite programs arising from MAX CUT and COLORING , 1996, STOC '96.

[29]  Thomas M. Cover,et al.  Universal data compression and portfolio selection , 1996, Proceedings of 37th Conference on Foundations of Computer Science.

[30]  T. Cover.  Universal Portfolios , 1996 .

[31]  Avrim Blum,et al.  On-line Algorithms in Machine Learning , 1996, Online Algorithms.

[32]  Yoram Singer,et al.  On‐Line Portfolio Selection Using Multiplicative Updates , 1998, ICML.

[33]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[34]  Avi Wigderson,et al.  Theory of computing , 1997, SIGACT News.

[35]  David Haussler,et al.  Sequential Prediction of Individual Sequences Under General Loss Functions , 1998, IEEE Trans. Inf. Theory.

[36]  U. Feige.  A threshold of ln n for approximating set cover , 1998, JACM.

[37]  Jochen Könemann,et al.  Faster and simpler algorithms for multicommodity flow and other fractional packing problems , 1998, Proceedings 39th Annual Symposium on Foundations of Computer Science (Cat. No.98CB36280).

[38]  Y. Freund,et al.  Adaptive game playing using multiplicative weights , 1999 .

[39]  Philip N. Klein,et al.  On the Number of Iterations for Dantzig-Wolfe Optimization and Packing-Covering Approximation Algorithms , 1999, SIAM J. Comput..

[40]  Dean P. Foster,et al.  Regret in the On-Line Decision Problem , 1999 .

[41]  Bernard Chazelle,et al.  The discrepancy method - randomness and complexity , 2000 .

[42]  Lisa Fleischer,et al.  Approximating Fractional Multicommodity Flow Independent of the Number of Commodities , 2000, SIAM J. Discret. Math..

[43]  Fernando Paganini,et al.  Internet congestion control , 2002 .

[44]  Noga Alon,et al.  The online set cover problem , 2003, STOC '03.

[45]  Santosh S. Vempala,et al.  Efficient algorithms for online decision problems , 2005, J. Comput. Syst. Sci..

[46]  Martin Zinkevich,et al.  Online Convex Programming and Generalized Infinitesimal Gradient Ascent , 2003, ICML.

[47]  Mark Herbster,et al.  Tracking the Best Expert , 1995, Machine Learning.

[48]  Rocco A. Servedio,et al.  Boosting and Hard-Core Set Construction , 2003, Machine Learning.

[49]  Elad Hazan,et al.  O(√log n) approximation to SPARSEST CUT in Õ(n 2) time , 2004, IEEE Annual Symposium on Foundations of Computer Science.

[50]  R. Khandekar.  Lagrangian relaxation based algorithms for convex programming problems , 2004 .

[51]  Naveen Garg,et al.  Fractional Covering with Upper Bounds on the Variables: Solving LPs with Negative Entries , 2004, ESA.

[52]  Gunnar Rätsch,et al.  Matrix Exponentiated Gradient Updates for On-line Learning and Bregman Projection , 2004, J. Mach. Learn. Res..

[53]  Robert Krauthgamer,et al.  Measured descent: a new embedding method for finite metrics , 2004, 45th Annual IEEE Symposium on Foundations of Computer Science.

[54]  Satish Rao,et al.  Expander flows, geometric embeddings and graph partitioning , 2004, STOC '04.

[55]  R. Schapire.  The Strength of Weak Learnability , 1990, Machine Learning.

[56]  Sanjeev Arora,et al.  Fast algorithms for approximate semidefinite programming using the multiplicative weights update method , 2005, 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05).

[57]  Anupam Gupta,et al.  Embeddings of negative-type metrics and an improved approximation to generalized sparsest cut , 2005, SODA '05.

[58]  James R. Lee,et al.  Euclidean distortion and the sparsest cut , 2005, STOC '05.

[59]  Elad Hazan,et al.  Efficient Algorithms for Online Game Playing and Universal Portfolio Management , 2006, Electron. Colloquium Comput. Complex..

[60]  Adam Tauman Kalai,et al.  Logarithmic Regret Algorithms for Online Convex Optimization , 2006, COLT.

[61]  Gábor Lugosi,et al.  Prediction, learning, and games , 2006 .

[62]  Yishay Mansour,et al.  Improved second-order bounds for prediction with expert advice , 2006, Machine Learning.

[63]  Sanjeev Arora,et al.  Efficient algorithms for online convex optimization and their applications , 2006 .

[64]  Sanjeev Arora,et al.  A combinatorial, primal-dual approach to semidefinite programs , 2007, STOC '07.

[65]  Satyen Kale,et al.  Boosting and hard-core set constructions: a simplified approach , 2007, Electron. Colloquium Comput. Complex..

[66]  Satyen Kale.  Efficient algorithms using the multiplicative weights update method , 2007 .

[67]  Elad Hazan,et al.  Logarithmic regret algorithms for online convex optimization , 2006, Machine Learning.

[68]  Sanjeev Arora,et al.  Euclidean distortion and the sparsest cut , 2005, Symposium on the Theory of Computing.

[69]  Elad Hazan,et al.  Sparse Approximate Solutions to Semidefinite Programs , 2008, LATIN.

[70]  Joseph Naor,et al.  The Design of Competitive Online Algorithms via a Primal-Dual Approach , 2009, Found. Trends Theor. Comput. Sci..

[71]  Joseph Naor,et al.  Online Primal-Dual Algorithms for Covering and Packing , 2009, Math. Oper. Res..

[72]  Noga Alon,et al.  The online set cover problem , 2003, STOC '03.

[73]  Boaz Barak,et al.  The uniform hardcore lemma via approximate Bregman projections , 2009, SODA.

[74]  Rahul Jain,et al.  QIP = PSPACE , 2010, STOC '10.

[75]  Manfred K. Warmuth,et al.  Online variance minimization , 2011, Machine Learning.