Duality and convex programming

This chapter surveys key concepts in convex duality theory and their application to the analysis and numerical solution of problem archetypes in imaging.

[1]  B. N. Pshenichnyi.  Necessary Conditions for an Extremum , 1971 .

[2]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[3]  Madhur Tulsiani,et al.  Convex Relaxations and Integrality Gaps , 2012 .

[4]  Marc Teboulle,et al.  A Linearly Convergent Algorithm for Solving a Class of Nonconvex/Affine Feasibility Problems , 2011, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[5]  Gene H. Golub,et al.  A Nonlinear Primal-Dual Method for Total Variation-Based Image Restoration , 1999, SIAM J. Sci. Comput..

[6]  R. Goodrich,et al.  $L_2 $ Spectral Estimation , 1986 .

[7]  Yves Lucet,et al.  Fast Moreau envelope computation I: numerical algorithms , 2007, Numerical Algorithms.

[8]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[9]  Yves Lucet,et al.  What Shape Is Your Conjugate? A Survey of Computational Convex Analysis and Its Applications , 2009, SIAM J. Optim..

[10]  R. Rockafellar,et al.  On the maximal monotonicity of subdifferential mappings. , 1970 .

[11]  J. P. Burg,et al.  Maximum entropy spectral analysis. , 1967 .

[12]  J. Tropp.  Algorithms for simultaneous sparse approximation. Part II: Convex relaxation , 2006, Signal Process..

[13]  R. Rockafellar.  Characterization of the subdifferentials of convex functions , 1966 .

[14]  S. Simons.  From Hahn-Banach to monotonicity , 2008 .

[15]  C. W. Groetsch,et al.  The theory of Tikhonov regularization for Fredholm equations of the first kind , 1984 .

[16]  Michael Hintermüller,et al.  An Infeasible Primal-Dual Algorithm for Total Bounded Variation-Based Inf-Convolution-Type Image Restoration , 2006, SIAM J. Sci. Comput..

[17]  I. Singer,et al.  Duality for Nonconvex Approximation and Optimization , 2010 .

[18]  D. Varberg.  Convex Functions , 1973 .

[19]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[20]  Stephen P. Boyd,et al.  CVXGEN: a code generator for embedded convex optimization , 2011, Optimization and Engineering.

[21]  R. Rockafellar.  Integrals which are convex functionals. II , 1968 .

[22]  H. Trussell,et al.  Method of successive projections for finding a common point of sets in metric spaces , 1990 .

[23]  P. L. Combettes,et al.  The Convex Feasibility Problem in Image Recovery , 1996 .

[24]  Heinz H. Bauschke,et al.  Fixed-Point Algorithms for Inverse Problems in Science and Engineering , 2011, Springer Optimization and Its Applications.

[25]  Y. Censor,et al.  Parallel Optimization: Theory, Algorithms, and Applications , 1997 .

[26]  Stephen E. Wright,et al.  A simple and elementary proof of the Karush–Kuhn–Tucker theorem for inequality-constrained optimization , 2009, Optim. Lett..

[27]  J. Borwein,et al.  Convex Analysis And Nonlinear Optimization , 2000 .

[28]  Otmar Scherzer,et al.  Variational Methods in Imaging , 2008, Applied mathematical sciences.

[29]  G. Minty.  Monotone (nonlinear) operators in Hilbert space , 1962 .

[30]  Heinz H. Bauschke,et al.  On Projection Algorithms for Solving Convex Feasibility Problems , 1996, SIAM Rev..

[31]  Stephen J. Wright.  Primal-Dual Interior-Point Methods , 1997, Other Titles in Applied Mathematics.

[32]  Charles L. Byrne,et al.  Signal Processing: A Mathematical Approach , 1993 .

[33]  Gilles Aubert,et al.  Efficient Schemes for Total Variation Minimization Under Constraints in Image Processing , 2009, SIAM J. Sci. Comput..

[34]  J. Borwein,et al.  Techniques of variational analysis , 2005 .

[35]  Stephen P. Boyd,et al.  Graph Implementations for Nonsmooth Convex Programs , 2008, Recent Advances in Learning and Control.

[36]  E. H. Zarantonello.  Projections on Convex Sets in Hilbert Space and Spectral Theory: Part I. Projections on Convex Sets: Part II. Spectral Theory , 1971 .

[37]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[38]  Adrian S. Lewis,et al.  Local Linear Convergence for Alternating and Averaged Nonconvex Projections , 2009, Found. Comput. Math..

[39]  I. Johnstone,et al.  Ideal spatial adaptation by wavelet shrinkage , 1994 .

[40]  F. Deutsch.  Best approximation in inner product spaces , 2001 .

[41]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[42]  Heinz H. Bauschke,et al.  Phase retrieval, error reduction algorithm, and Fienup variants: a view from convex optimization. , 2002, Journal of the Optical Society of America. A, Optics, image science, and vision.

[43]  Jonathan M. Borwein,et al.  Convergence of Best Entropy Estimates , 1991, SIAM J. Optim..

[44]  Antonin Chambolle,et al.  A l1-Unified Variational Framework for Image Restoration , 2004, ECCV.

[45]  Jonathan M. Borwein,et al.  Maximum Entropy Reconstruction Using Derivative Information, Part 1: Fisher Information and Convex Duality , 1996, Math. Oper. Res..

[46]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[47]  Joel A. Tropp,et al.  ALGORITHMS FOR SIMULTANEOUS SPARSE APPROXIMATION , 2006 .

[48]  L. Levi,et al.  Fitting a bandlimited signal to given points , 1965, IEEE Trans. Inf. Theory.

[49]  C. Zălinescu.  Convex analysis in general vector spaces , 2002 .

[50]  J. Borwein,et al.  Convex Functions: Constructions, Characterizations and Counterexamples , 2010 .

[51]  B. Svaiter,et al.  A new proof for maximal monotonicity of subdierential operators , 2006 .

[52]  Yves Lucet,et al.  Faster than the Fast Legendre Transform, the Linear-time Legendre Transform , 1997, Numerical Algorithms.

[53]  Roberto Lucchetti,et al.  Convexity and well-posed problems , 2006 .

[54]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[55]  P. P. B. Eggermont,et al.  Maximum entropy regularization for Fredholm integral equations of the first kind , 1993 .

[56]  Mike E. Davies,et al.  Normalized Iterative Hard Thresholding: Guaranteed Stability and Performance , 2010, IEEE Journal of Selected Topics in Signal Processing.

[57]  I. Ekeland,et al.  Convex analysis and variational problems , 1976 .

[58]  Rachid Deriche,et al.  Regularizing Flows for Constrained Matrix-Valued Images , 2004 .

[59]  Heinz H. Bauschke,et al.  The piecewise linear-quadratic model for computational convex analysis , 2009, Comput. Optim. Appl..

[60]  N I Grinberg,et al.  The Factorization Method for Inverse Problems , 2007 .

[61]  Jonathan M. Borwein,et al.  Symbolic Fenchel Conjugation , 2008, Math. Program..

[62]  Marc Teboulle,et al.  Convergence of best phi-entropy estimates , 1993, IEEE Trans. Inf. Theory.

[63]  J. Borwein,et al.  Maximum entropy reconstruction using derivative information part 2: computational results , 1995 .

[64]  W. Fenchel.  On Conjugate Convex Functions , 1949, Canadian Journal of Mathematics.

[65]  M. Nikolova.  An Algorithm for Total Variation Minimization and Applications , 2004 .

[66]  Yu. S. Ledyaev,et al.  Nonsmooth analysis and control theory , 1998 .

[67]  Panagiotis Patrinos,et al.  Convex parametric piecewise quadratic optimization: Theory and algorithms , 2011, Autom..

[68]  Adrian S. Lewis,et al.  Alternating Projections on Manifolds , 2008, Math. Oper. Res..

[69]  Simon Foucart,et al.  Hard Thresholding Pursuit: An Algorithm for Compressive Sensing , 2011, SIAM J. Numer. Anal..

[70]  B. Mordukhovich.  Variational analysis and generalized differentiation , 2006 .

[71]  Stephen P. Boyd,et al.  Recent Advances in Learning and Control , 2008, Lecture Notes in Control and Information Sciences.

[72]  J. Borwein,et al.  Duality relationships for entropy-like minimization problems , 1991 .

[73]  R. Phelps.  Convex Functions, Monotone Operators and Differentiability , 1989 .

[74]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[75]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[76]  J. Craggs.  Applied Mathematical Sciences , 1973 .

[77]  L. Potter,et al.  A dual approach to linear inverse problems with convex constraints , 1993 .

[78]  Marc Teboulle,et al.  A dual approach to multidimensional L p spectral estimation problems , 1988 .

[79]  Jonathan M. Borwein,et al.  On the failure of maximum entropy reconstruction for Fredholm equations and other infinite systems , 1993, Math. Program..

[80]  R. Kress.  Linear Integral Equations , 1989 .

[81]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[82]  D. Russell Luke,et al.  Finding Best Approximation Pairs Relative to a Convex and Prox-Regular Set in a Hilbert Space , 2008, SIAM J. Optim..

[83]  K. Ilk.  On the Regularization of Ill-Posed Problems , 1987 .

[84]  J. Hiriart-Urruty,et al.  Fundamentals of Convex Analysis , 2004 .

[85]  R. Rockafellar.  Conjugate Duality and Optimization , 1987 .

[86]  Didier Dacunha-Castelle,et al.  Maximum d'entropie et problème des moments , 1990 .

[87]  Marc Teboulle,et al.  A regularized dual-based iterative method for a class of image reconstruction problems , 1993 .

[88]  P. L. Combettes,et al.  Dualization of Signal Recovery Problems , 2009, 0907.0436.

[89]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[90]  Charles W. Groetsch,et al.  Stable Approximate Evaluation of Unbounded Operators , 2006 .

[91]  P. Lions,et al.  Image recovery via total variation minimization and related problems , 1997 .

[92]  M. Jaoua,et al.  A dual algorithm for denoising and preserving edges in image processing , 2007 .

[93]  Mike E. Davies,et al.  Iterative Hard Thresholding for Compressed Sensing , 2008, ArXiv.

[94]  Heinz H. Bauschke,et al.  Hybrid projection-reflection method for phase retrieval. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[95]  Joel A. Tropp,et al.  Just relax: convex programming methods for identifying sparse signals in noise , 2006, IEEE Transactions on Information Theory.

[96]  Pierre Maréchal,et al.  Unification of some deterministic and probabilistic methods for the solution of linear inverse problems via the principle of maximum entropy on the mean , 1997 .

[97]  D. R. Luke.  Relaxed Averaged Alternating Reflections for Diffraction Imaging , 2004, math/0405208.

[98]  J. Moreau.  Fonctions convexes duales et points proximaux dans un espace hilbertien , 1962 .

[99]  James V. Burke,et al.  Optical Wavefront Reconstruction: Theory and Numerical Methods , 2002, SIAM Rev..

[100]  Victor Klee,et al.  Convexity of Chebyshev sets , 1961 .

[101]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[102]  J. Moreau.  Proximité et dualité dans un espace hilbertien , 1965 .

[103]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.

[104]  David G. Luenberger,et al.  Linear and nonlinear programming , 1984 .

[105]  Jonathan M. Borwein,et al.  Entropic Regularization of the ℓ 0 Function , 2011, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[106]  Jonathan M. Borwein,et al.  The Douglas-Rachford Algorithm in the Absence of Convexity , 2011, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[107]  Y. Lucet.  Hybrid symbolic‐numeric algorithms for computational convex analysis , 2007 .

[108]  Heinz H. Bauschke,et al.  Finding best approximation pairs relative to two closed convex sets in Hilbert spaces , 2004, J. Approx. Theory.

[109]  M. Teboulle,et al.  Asymptotic cones and functions in optimization and variational inequalities , 2002 .

[110]  James V. Burke,et al.  Variational Analysis Applied to the Problem of Optical Phase Retrieval , 2003, SIAM J. Control. Optim..