Convex programming with fast proximal and linear operators

We present Epsilon, a system for general convex programming using fast linear and proximal operators. As with existing convex programming frameworks, users specify convex optimization problems using a natural grammar for mathematical expressions, composing functions in a way that is guaranteed to be convex by the rules of disciplined convex programming. Given such an input, the Epsilon compiler transforms the optimization problem into a mathematically equivalent form consisting only of functions with efficient proximal operators---an intermediate representation we refer to as prox-affine form. By reducing problems to this form, Epsilon enables solving general convex problems using a large library of fast proximal and linear operators; numerical examples on many popular problems from statistics and machine learning show that this often improves running times by an order of magnitude or more vs. existing approaches based on conic solvers.

[1]  Wen Gao,et al.  Efficient Generalized Fused Lasso and its Application to the Diagnosis of Alzheimer's Disease , 2014, AAAI.

[2]  Stephen P. Boyd,et al.  CVXPY: A Python-Embedded Modeling Language for Convex Optimization , 2016, J. Mach. Learn. Res..

[3]  Stephen P. Boyd,et al.  ECOS: An SOCP solver for embedded systems , 2013, 2013 European Control Conference (ECC).

[4]  R. Tibshirani,et al.  Sparsity and smoothness via the fused lasso , 2005 .

[5]  Stephen P. Boyd,et al.  Convex Optimization in Julia , 2014, 2014 First Workshop for High Performance Technical Computing in Dynamic Languages.

[6]  Damek Davis,et al.  Convergence Rate Analysis of Several Splitting Schemes , 2014, 1406.4834.

[7]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[8]  Pontus Giselsson,et al.  Tight global linear convergence rate bounds for Douglas–Rachford splitting , 2015, Journal of Fixed Point Theory and Applications.

[9]  Stephen P. Boyd,et al.  Disciplined Convex Programming , 2006 .

[10]  Stephen P. Boyd,et al.  Conic Optimization via Operator Splitting and Homogeneous Self-Dual Embedding , 2013, Journal of Optimization Theory and Applications.

[11]  Johan Löfberg,et al.  YALMIP : a toolbox for modeling and optimization in MATLAB , 2004 .

[12]  Stephen P. Boyd,et al.  A Primer on Monotone Operator Methods , 2015 .

[13]  Michael I. Jordan,et al.  A General Analysis of the Convergence of ADMM , 2015, ICML.

[14]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[15]  Yoram Singer,et al.  Efficient projections onto the l1-ball for learning in high dimensions , 2008, ICML '08.

[16]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[17]  J. Moreau.  Fonctions convexes duales et points proximaux dans un espace hilbertien , 1962 .

[18]  Stephen J. Wright,et al.  Primal-Dual Interior-Point Methods , 1997 .

[19]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[20]  Stephen P. Boyd,et al.  Convex Optimization with Abstract Linear Operators , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[21]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[22]  Ryan J. Tibshirani,et al.  Fast and Flexible ADMM Algorithms for Trend Filtering , 2014, ArXiv.

[23]  Nicholas A. Johnson,et al.  A Dynamic Programming Algorithm for the Fused Lasso and L 0-Segmentation , 2013 .