One-sided Frank-Wolfe algorithms for saddle problems

We study a class of convex-concave saddlepoint problems of the form minx maxy〈Kx, y〉+ fP(x)−h(y) whereK is a linear operator, fP is the sum of a convex function f with a Lipschitzcontinuous gradient and the indicator function of a bounded convex polytope P , and h∗ is a convex (possibly nonsmooth) function. Such problem arises, for example, as a Lagrangian relaxation of various discrete optimization problems. Our main assumptions are the existence of an efficient linear minimization oracle (lmo) for fP and an efficient proximal map (prox) for h∗ which motivate the solution via a blend of proximal primaldual algorithms and Frank-Wolfe algorithms. In case h∗ is the indicator function of a linear constraint and function f is quadratic, we show a O(1/n) convergence rate on the dual objective, requiring O(n log n) calls of lmo. If the problem comes from the constrained optimization problem minx∈Rd{fP(x) |Ax− b = 0} then we additionally get bound O(1/n) both on the primal gap and on the infeasibility gap. In the most general case, we show a O(1/n) convergence rate of the primal-dual gap again requiring O(n log n) calls of lmo. To the best of our knowledge, this improves on the known convergence rates for the considered class of saddle-point problems. We show applications to labeling problems frequently appearing in machine learning and computer vision.