A stochastic proximal point algorithm for total variation regularization over large scale graphs

The total-variation (TV) regularizer is often used to promote the structured sparsity of a given real function over the vertices of a non-directed graph. Indeed, the proximity operator associated with TV regularizer promotes sparsity of the function discrete gradient. Although quite affordable in the special case of one-dimensional (1D) graphs, the computation of the proximity operator for general large scale graphs can be demanding. In this paper, we propose a stochastic algorithm for solving this problem over large graphs with a moderate iteration complexity. The algorithm consists in properly selecting random paths in the graph and computing 1D-proximity operators over these paths. Convergence of the algorithm is related to recent results on stochastic proximal point algorithms.

[1]  S. Geer,et al.  Locally adaptive regression splines , 1997 .

[2]  D. Bertsekas,et al.  Incremental Constraint Projection-Proximal Methods for Nonsmooth Convex Optimization , 2013 .

[3]  James G. Scott,et al.  A Fast and Flexible Algorithm for the Graph-Fused Lasso , 2015, 1505.06475.

[4]  J. Hartigan,et al.  The Dip Test of Unimodality , 1985 .

[5]  Suvrit Sra,et al.  Modular Proximal Optimization for Multidimensional Total-Variation Regularization , 2014, J. Mach. Learn. Res..

[6]  Pascal Bianchi,et al.  Dynamical Behavior of a Stochastic Forward–Backward Algorithm Using Random Monotone Operators , 2015, J. Optim. Theory Appl..

[7]  Markus Grasmair,et al.  The Equivalence of the Taut String Algorithm and BV-Regularization , 2006, Journal of Mathematical Imaging and Vision.

[8]  R. Tibshirani.  Adaptive piecewise polynomial estimation via trend filtering , 2013, 1304.2986.

[9]  P. L. Combettes.  Iterative construction of the resolvent of a sum of maximal monotone operators , 2009 .

[10]  H. D. Brunk,et al.  Statistical inference under order restrictions : the theory and application of isotonic regression , 1973 .

[11]  Kathryn B. Laskey,et al.  Stochastic blockmodels: First steps , 1983 .

[12]  A. Chambolle,et al.  An introduction to Total Variation for Image Analysis , 2009 .

[13]  Suvrit Sra,et al.  Fast Newton-type Methods for Total Variation Regularization , 2011, ICML.

[14]  Alexander J. Smola,et al.  Trend Filtering on Graphs , 2014, J. Mach. Learn. Res..

[15]  Laurent Condat,et al.  A Direct Algorithm for 1-D Total Variation Denoising , 2013, IEEE Signal Processing Letters.

[16]  Pascal Bianchi,et al.  Ergodic Convergence of a Stochastic Proximal Point Algorithm , 2015, SIAM J. Optim..

[17]  Pascal Bianchi,et al.  Robust Distributed Consensus Using Total Variation , 2016, IEEE Transactions on Automatic Control.

[18]  R. Rockafellar.  Measurable dependence of convex sets and functions on parameters , 1969 .

[19]  Suvrit Sra,et al.  Reflection methods for user-friendly submodular optimization , 2013, NIPS.

[20]  Jorge Nocedal,et al.  A Limited Memory Algorithm for Bound Constrained Optimization , 1995, SIAM J. Sci. Comput..

[21]  Z. Harchaoui,et al.  Multiple Change-Point Estimation With a Total Variation Penalty , 2010 .

[22]  Stephen P. Boyd,et al.  1 Trend Filtering , 2009, SIAM Rev..

[23]  Stephan Didas,et al.  Relations Between Higher Order TV Regularization and Support Vector Regression , 2005, Scale-Space.

[24]  R. Tibshirani,et al.  The solution path of the generalized lasso , 2010, 1005.1971.

[25]  P. Davies,et al.  Local Extremes, Runs, Strings and Multiresolution , 2001 .

[26]  Gregory B. Passty.  Ergodic convergence to a zero of the sum of monotone operators in Hilbert space , 1979 .

[27]  Otmar Scherzer,et al.  Tube Methods for BV Regularization , 2003, Journal of Mathematical Imaging and Vision.

[28]  Alexander Shapiro,et al.  Stochastic Approximation approach to Stochastic Programming , 2013 .