ECLIPSE: An Extreme-Scale Linear Program Solver for Web-Applications

Key problems arising in web applications (with millions of users and thousands of items) can be formulated as Linear Programs (LP) involving billions to trillions of decision variables and constraints. Despite the appeal of LP formulations, solving problems at these scales appears to be well beyond the capabilities of existing LP solvers. Often ad-hoc decomposition rules are used to approximately solve these LPs, which have limited optimality guarantees and lead to a sub-optimal performance in practice. In this work, we propose a distributed solver that solves a perturbation of the LP problems at scale via a gradient-based algorithm on the smooth dual of the perturbed LP with computational guarantees. The main workhorses of our algorithm are distributed matrix-vector multiplications (with load balancing) and efficient projection operations on distributed machines. Experiments on real-world data show that our proposed LP solver, ECLIPSE, can solve problems with 10 decision variables – well beyond the capabilities of current solvers.

[1]  Liang Zhang,et al.  LiJAR: A System for Job Application Redistribution towards Efficient Career Marketplace , 2017, KDD.

[2]  Rómer Rosales,et al.  Optimizing Email Volume For Sitewide Engagement , 2017, CIKM.

[3]  Deepak Agarwal,et al.  Personalized click shaping through lagrangian duality for online recommendation , 2012, SIGIR '12.

[4]  O. Mangasarian,et al.  NONLINEAR PERTURBATION OF LINEAR PROGRAMS , 1979 .

[5]  Stephen P. Boyd,et al.  Conic Optimization via Operator Splitting and Homogeneous Self-Dual Embedding , 2013, Journal of Optimization Theory and Applications.

[6]  Liang Zhang,et al.  Activity ranking in LinkedIn feed , 2014, KDD.

[7]  Jasper Snoek,et al.  Practical Bayesian Optimization of Machine Learning Algorithms , 2012, NIPS.

[8]  Deepak Agarwal,et al.  Click shaping to optimize multiple objectives , 2011, KDD.

[9]  Shuyang Du,et al.  Improve User Retention with Causal Learning , 2019, CD@KDD.

[10]  Xiaoming Yuan,et al.  Adaptive Primal-Dual Hybrid Gradient Methods for Saddle-Point Problems , 2013, 1305.0546.

[11]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[12]  Ganesh Venkataraman,et al.  Personalized Job Recommendation System at LinkedIn: Practical Challenges and Lessons Learned , 2017, RecSys.

[13]  Jacek Gondzio,et al.  Parallel interior-point solver for structured linear programs , 2003, Math. Program..

[14]  Yurii Nesterov,et al.  Gradient methods for minimizing composite functions , 2012, Mathematical Programming.

[15]  E. M. Azevedo,et al.  Matching markets in the digital age , 2016, Science.

[16]  Guilherme Ottoni,et al.  Constrained Bayesian Optimization with Noisy Experiments , 2017, Bayesian Analysis.

[17]  Qi He,et al.  Personalizing LinkedIn Feed , 2015, KDD.

[18]  Jasbir S. Arora,et al.  Survey of multi-objective optimization methods for engineering , 2004 .

[19]  Xiaoyu Chen,et al.  Email Volume Optimization at LinkedIn , 2016, KDD.

[20]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[21]  Paul Tseng,et al.  Exact Regularization of Convex Programs , 2007, SIAM J. Optim..

[22]  Bingsheng He,et al.  The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent , 2014, Mathematical Programming.

[23]  Stephen P. Boyd,et al.  Block splitting for distributed optimization , 2013, Mathematical Programming Computation.

[24]  Bin Dong,et al.  Fast Linearized Bregman Iteration for Compressive Sensing and Sparse Denoising , 2011, ArXiv.

[25]  Robert E. Bixby,et al.  Solving Real-World Linear Programs: A Decade and More of Progress , 2002, Oper. Res..

[26]  John N. Tsitsiklis,et al.  Introduction to linear optimization , 1997, Athena scientific optimization and computation series.

[27]  Jie Wu,et al.  Online to Offline Business: Urban Taxi Dispatching with Passenger-Driver Matching Stability , 2017, 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS).

[28]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .

[29]  Yurii Nesterov,et al.  Smooth minimization of non-smooth functions , 2005, Math. Program..

[30]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[31]  Ameet Talwalkar,et al.  MLlib: Machine Learning in Apache Spark , 2015, J. Mach. Learn. Res..

[32]  Greg Linden,et al.  Amazon . com Recommendations Item-to-Item Collaborative Filtering , 2001 .

[33]  Rupak Biswas,et al.  High performance computing using MPI and OpenMP on multi-core parallel systems , 2011, Parallel Comput..

[34]  Kinjal Basu,et al.  Online Parameter Selection for Web-based Ranking Problems , 2018, KDD.

[35]  Justin David Durfee,et al.  Comparison of open-source linear programming solvers. , 2013 .

[36]  Antonin Chambolle,et al.  Diagonal preconditioning for first order primal-dual algorithms in convex optimization , 2011, 2011 International Conference on Computer Vision.

[37]  Shipeng Yu,et al.  Internal Promotion Optimization , 2019, KDD.

[38]  Eytan Bakshy,et al.  Bayesian Optimization for Policy Search via Online-Offline Experimentation , 2019, J. Mach. Learn. Res..

[39]  Yan Gao,et al.  Near Real-time Optimization of Activity-based Notifications , 2018, KDD.

[40]  A. M. Madni,et al.  Recommender systems in e-commerce , 2014, 2014 World Automation Congress (WAC).

[41]  Rómer Rosales,et al.  A State Transition Model for Mobile Notifications via Survival Analysis , 2019, WSDM.