Ecient and Practical Stochastic Subgradient Descent for Nuclear

We describe novel subgradient methods for a broad class of matrix optimization problems involving nuclear norm regularization. Unlike existing approaches, our method executes very cheap iterations by combining low-rank stochastic subgradients with ecient incremental SVD updates, made possible by highly optimized and parallelizable dense linear algebra operations on small matrices. Our practical algorithms always maintain a low-rank factorization of iterates that can be conveniently held in memory and efciently multiplied to generate predictions in matrix completion settings. Empirical comparisons conrm that our approach is highly competitive with several recently proposed state-of-the-art solvers for such problems.