On the acceleration of the double smoothing technique for unconstrained convex optimization problems

In this article, we investigate the possibilities of accelerating the double smoothing (DS) technique when solving unconstrained nondifferentiable convex optimization problems. This approach relies on the regularization in two steps of the Fenchel dual problem associated with the problem to be solved into an optimization problem having a differentiable strongly convex objective function with Lipschitz continuous gradient. The doubly regularized dual problem is then solved via a fast gradient method. The aim of this article is to show how the properties of the functions in the objective of the primal problem influence the implementation of the DS approach and its rate of convergence. The theoretical results are applied to linear inverse problems by making use of different regularization functionals.

[1]  Radu Ioan Bot,et al.  A double smoothing technique for solving unconstrained nondifferentiable convex optimization problems , 2012, Computational Optimization and Applications.

[2]  R. Boţ,et al.  Iterative regularization with a general penalty term—theory and application to L1 and TV regularization , 2012 .

[3]  Yurii Nesterov,et al.  Double Smoothing Technique for Large-Scale Linearly Constrained Convex Optimization , 2012, SIAM J. Optim..

[4]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[5]  Y. Nesterov,et al.  A DOUBLE SMOOTHING TECHNIQUE FOR CONSTRAINED CONVEX OPTIMIZATION PROBLEMS AND APPLICATIONS TO OPTIMAL CONTROL , 2011 .

[6]  Y. Nesterov,et al.  Double smoothing technique for infinite-dimensional optimization problems with applications to optimal control , 2010 .

[7]  R. Boţ,et al.  Conjugate Duality in Convex Optimization , 2010 .

[8]  Marc Teboulle,et al.  Gradient-based algorithms with applications to signal-recovery problems , 2010, Convex Optimization in Signal Processing and Communications.

[9]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[10]  Yurii Nesterov,et al.  Smoothing Technique and its Applications in Semidefinite Optimization , 2004, Math. Program..

[11]  Yurii Nesterov,et al.  Excessive Gap Technique in Nonsmooth Convex Minimization , 2005, SIAM J. Optim..

[12]  Yurii Nesterov,et al.  Smooth minimization of non-smooth functions , 2005, Math. Program..

[13]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[14]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[15]  Johannes Jahn,et al.  Duality in vector optimization , 1983, Math. Program..