Foundations of Gauge and Perspective Duality

We revisit the foundations of gauge duality and demonstrate that it can be explained using a modern approach to duality based on a perturbation framework. We therefore put gauge duality and Fenchel-Rockafellar duality on equal footing, including explaining gauge dual variables as sensitivity measures, and showing how to recover primal solutions from those of the gauge dual. This vantage point allows a direct proof that optimal solutions of the Fenchel-Rockafellar dual of the gauge dual are precisely the primal solutions rescaled by the optimal value. We extend the gauge duality framework to the setting in which the functional components are general nonnegative convex functions, including problems with piecewise linear quadratic functions and constraints that arise from generalized linear models used in regression.

[1]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[2]  R. Tyrrell Rockafellar,et al.  Convex Analysis , 1970, Princeton Landmarks in Mathematics and Physics.

[3]  Robert M. Freund,et al.  Dual gauge programs, with applications to quadratic programming and the minimum-norm problem , 1987, Math. Program..

[4]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[5]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[6]  R. Rockafellar.  First- and second-order epi-differentiability in nonlinear programming , 1988 .

[7]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[8]  Dmitriy Drusvyatskiy,et al.  Level-set methods for convex optimization , 2016, Mathematical Programming.

[9]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.

[10]  Michael P. Friedlander,et al.  Gauge Optimization and Duality , 2013, SIAM J. Optim..

[11]  Michael P. Friedlander,et al.  Low-Rank Spectral Optimization via Gauge Duality , 2015, SIAM J. Sci. Comput..

[12]  Marc Teboulle,et al.  On the rate of convergence of the proximal alternating linearized minimization algorithm for convex problems , 2016, EURO J. Comput. Optim..

[13]  R. Tyrrell Rockafellar,et al.  Variational Analysis , 1998, Grundlehren der mathematischen Wissenschaften.

[14]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[15]  Inderjit S. Dhillon,et al.  Matrix Nearness Problems with Bregman Divergences , 2007, SIAM J. Matrix Anal. Appl..

[16]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[17]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.

[18]  Stephen P. Boyd,et al.  Convex Optimization in Julia , 2014, 2014 First Workshop for High Performance Technical Computing in Dynamic Languages.

[19]  Adrian S. Lewis,et al.  Clarke Subgradients of Stratifiable Functions , 2006, SIAM J. Optim..

[20]  Aleksandr Y. Aravkin,et al.  Linear system identification using stable spline kernels and PLQ penalties , 2013, 52nd IEEE Conference on Decision and Control.

[21]  Michael P. Friedlander,et al.  Variational Properties of Value Functions , 2012, SIAM J. Optim..

[22]  Inderjit S. Dhillon,et al.  Clustering with Bregman Divergences , 2005, J. Mach. Learn. Res..

[23]  Joel A. Tropp,et al.  Just relax: convex programming methods for identifying sparse signals in noise , 2006, IEEE Transactions on Information Theory.

[24]  Patrick L. Combettes,et al.  Perspective Functions: Proximal Calculus and Applications in High-Dimensional Statistics , 2016, 1610.01478.

[25]  P. McCullagh,et al.  Generalized Linear Models , 1984 .

[26]  Marc Teboulle,et al.  A dual method for minimizing a nonsmooth objective over one smooth inequality constraint , 2016, Math. Program..

[27]  Patrick L. Combettes,et al.  Perspective Functions: Properties, Constructions, and Examples , 2016, Set-Valued and Variational Analysis.

[28]  R. Rockafellar.  Conjugate Duality and Optimization , 1987 .

[29]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .