Efficiency of randomized coordinate descent methods on minimization problems with a composite objective function

We develop a randomized block-coordinate descent method for minimizing the sum of a smooth and a simple nonsmooth blockseparable convex function and prove that it obtains an ε-accurate solution with probability at least 1−ρ in at most O((4n/ε) log(1/ερ)) iterations, where n is the dimension of the problem. This extends recent results of Nesterov [2], which cover the smooth case, to composite minimization, and improves the complexity by a factor of 2. In the smooth case we give a much simplified analysis. Finally, we demonstrate numerically that the algorithm is able to solve various l1-regularized optimization problems with a billion variables.