An Improved Deterministic Rescaling for Linear Programming Algorithms

The perceptron algorithm for linear programming, arising from machine learning, has been around since the 1950s. While not a polynomial-time algorithm, it is useful in practice due to its simplicity and robustness. In 2004, Dunagan and Vempala showed that a randomized rescaling turns the perceptron method into a polynomial time algorithm, and later Pena and Soheili gave a deterministic rescaling. In this paper, we give a deterministic rescaling for the perceptron algorithm that improves upon the previous rescaling methods by making it possible to rescale much earlier. This results in a faster running time for the rescaled perceptron algorithm. We will also demonstrate that the same rescaling methods yield a polynomial time algorithm based on the multiplicative weights update method. This draws a connection to an area that has received a lot of recent attention in theoretical computer science.

[1]  Jochen Könemann,et al.  Faster and simpler algorithms for multicommodity flow and other fractional packing problems , 1998, Proceedings 39th Annual Symposium on Foundations of Computer Science (Cat. No.98CB36280).

[2]  David P. Williamson,et al.  The Design of Approximation Algorithms , 2011 .

[3]  L. Khachiyan.  Polynomial algorithms in linear programming , 1980 .

[4]  K. Ball.  An Elementary Introduction to Modern Convex Geometry , 1997 .

[5]  Sanjeev Arora,et al.  Fast algorithms for approximate semidefinite programming using the multiplicative weights update method , 2005, 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05).

[6]  Sergei Chubanov.  A strongly polynomial algorithm for linear systems having a binary solution , 2012, Math. Program..

[7]  Narendra Karmarkar,et al.  A new polynomial-time algorithm for linear programming , 1984, Comb..

[8]  H. P. Williams.  THEORY OF LINEAR AND INTEGER PROGRAMMING (Wiley-Interscience Series in Discrete Mathematics and Optimization) , 1989 .

[9]  F. John.  Extremum Problems with Inequalities as Subsidiary Conditions , 2014 .

[10]  Sanjeev Arora,et al.  The Multiplicative Weights Update Method: a Meta-Algorithm and Applications , 2012, Theory Comput..

[11]  Santosh S. Vempala,et al.  A simple polynomial-time rescaling algorithm for solving linear programs , 2004, STOC '04.

[12]  L. G. H. Cijan.  A polynomial algorithm in linear programming , 1979 .

[13]  J. A. Salvato.  John wiley & sons. , 1994, Environmental science & technology.

[14]  I. J. Schoenberg,et al.  The Relaxation Method for Linear Inequalities , 1954, Canadian Journal of Mathematics.

[15]  Shang-Hua Teng,et al.  Electrical flows, laplacian systems, and faster approximation of maximum flow in undirected graphs , 2010, STOC '11.

[16]  Noga Alon,et al.  The Probabilistic Method , 2015, Fundamentals of Ramsey Theory.

[17]  Éva Tardos,et al.  Fast approximation algorithms for fractional packing and covering problems , 1991, [1991] Proceedings 32nd Annual Symposium of Foundations of Computer Science.

[18]  Yurii Nesterov,et al.  Excessive Gap Technique in Nonsmooth Convex Minimization , 2005, SIAM J. Optim..

[19]  Sergei Chubanov,et al.  A polynomial projection algorithm for linear feasibility problems , 2015, Math. Program..

[20]  László A. Végh,et al.  Rescaling Algorithms for Linear Programming - Part I: Conic feasibility , 2016, ArXiv.

[21]  Aleksander Madry,et al.  Faster approximation schemes for fractional multicommodity flow problems via dynamic graph algorithms , 2010, STOC '10.

[22]  Javier Peña,et al.  A deterministic rescaled perceptron algorithm , 2016, Math. Program..

[23]  László A. Végh,et al.  Rescaled Coordinate Descent Methods for Linear Programming , 2016, IPCO.

[24]  Ulrich Betke.  Relaxation, New Combinatorial and Polynomial Algorithms for the Linear Feasibility Problem , 2004, Discret. Comput. Geom..

[25]  G. Nemhauser,et al.  Integer Programming , 2020 .

[26]  Vijay V. Vazirani,et al.  Approximation Algorithms , 2001, Springer Berlin Heidelberg.

[27]  V. Klee,et al.  HOW GOOD IS THE SIMPLEX ALGORITHM , 1970 .

[28]  Javier Peña,et al.  A Smooth Perceptron Algorithm , 2012, SIAM J. Optim..

[29]  J. Pach,et al.  Wiley‐Interscience Series in Discrete Mathematics and Optimization , 2011 .

[30]  Martin Skutella,et al.  Proceedings of the 18th International Conference on Integer Programming and Combinatorial Optimization - Volume 9682 , 2016 .