The Douglas–Rachford algorithm for convex and nonconvex feasibility problems

The Douglas–Rachford algorithm is an optimization method that can be used for solving feasibility problems. To apply the method, it is necessary that the problem at hand is prescribed in terms of constraint sets having efficiently computable nearest points. Although the convergence of the algorithm is guaranteed in the convex setting, the scheme has demonstrated to be a successful heuristic for solving combinatorial problems of different type. In this self-contained tutorial, we develop the convergence theory of projection algorithms within the framework of fixed point iterations, explain how to devise useful feasibility problem formulations, and demonstrate the application of the Douglas–Rachford method to said formulations. The paradigm is then illustrated on two concrete problems: a generalization of the “eight queens puzzle” known as the “( m ,  n )-queens problem”, and the problem of constructing a probability distribution with prescribed moments.

[1]  Guy Pierra,et al.  Decomposition through formalization in a product space , 1984, Math. Program..

[2]  Bishnu P. Lamichhane,et al.  APPLICATION OF PROJECTION ALGORITHMS TO DIFFERENTIAL EQUATIONS: BOUNDARY VALUE PROBLEMS , 2017, The ANZIAM Journal.

[3]  Benar Fux Svaiter,et al.  On Weak Convergence of the Douglas-Rachford Method , 2010, SIAM J. Control. Optim..

[4]  Jonathan M. Borwein,et al.  The Cyclic Douglas-Rachford Method for Inconsistent Feasibility Problems , 2013, 1310.2195.

[5]  Heinz H. Bauschke,et al.  On the Douglas–Rachford algorithm , 2016, Mathematical Programming.

[6]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[7]  D. Russell Luke,et al.  Quantitative Convergence Analysis of Iterated Expansive, Set-Valued Mappings , 2016, Math. Oper. Res..

[8]  Veit Elser.  Phase retrieval by iterated projections. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[9]  Nguyen Hieu Thao,et al.  A convergent relaxation of the Douglas–Rachford algorithm , 2017, Computational Optimization and Applications.

[10]  Jonathan M. Borwein,et al.  Reflection methods for inverse problems with applications to protein conformation determination , 2017 .

[11]  Roger Behling,et al.  Circumcentering the Douglas–Rachford method , 2017, Numerical Algorithms.

[12]  Matthew K. Tam.  Algorithms based on unions of nonexpansive maps , 2018, Optim. Lett..

[13]  Francisco J. Aragón Artacho,et al.  A new projection method for finding the closest point in the intersection of convex sets , 2016, Comput. Optim. Appl..

[14]  Heinz H. Bauschke,et al.  On the local convergence of the Douglas–Rachford algorithm , 2014, 1401.6188.

[15]  Andrzej Cegielski,et al.  Projection methods: an annotated bibliography of books and reviews , 2014, 1406.6143.

[16]  A. Cegielski.  Iterative Methods for Fixed Point Problems in Hilbert Spaces , 2012 .

[17]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[18]  Heinz H. Bauschke,et al.  The Douglas–Rachford algorithm for a hyperplane and a doubleton , 2018, J. Glob. Optim..

[19]  D. Russell Luke,et al.  Finding Best Approximation Pairs Relative to a Convex and Prox-Regular Set in a Hilbert Space , 2008, SIAM J. Optim..

[20]  Yair Censor,et al.  The cyclic Douglas–Rachford algorithm with r-sets-Douglas–Rachford operators , 2018, Optim. Methods Softw..

[21]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[22]  Matthew K. Tam,et al.  Union Averaged Operators with Applications to Proximal Algorithms for Min-Convex Functions , 2019, J. Optim. Theory Appl..

[23]  Matthew K. Tam,et al.  DOUGLAS–RACHFORD FEASIBILITY METHODS FOR MATRIX COMPLETION PROBLEMS , 2013, The ANZIAM Journal.

[24]  Y. Censor.  Iterative Methods for the Convex Feasibility Problem , 1984 .

[25]  Hung M. Phan,et al.  Linear convergence of the Douglas–Rachford method for two closed sets , 2014, 1401.6509.

[26]  øöö Blockinø.  Phase retrieval, error reduction algorithm, and Fienup variants: A view from convex optimization , 2002 .

[27]  Heinz H. Bauschke,et al.  The rate of linear convergence of the Douglas-Rachford algorithm for subspaces is the cosine of the Friedrichs angle , 2013, J. Approx. Theory.

[28]  Jonathan M. Borwein,et al.  The Douglas-Rachford Algorithm in the Absence of Convexity , 2011, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[29]  T.Wang,et al.  A GENERALIZATION OF THE n-QUEEN PROBLEM , 1989 .

[30]  Jonathan M. Borwein,et al.  Recent Results on Douglas–Rachford Methods for Combinatorial Optimization Problems , 2013, J. Optim. Theory Appl..

[31]  J. Borwein,et al.  Convex Analysis And Nonlinear Optimization , 2000 .

[32]  F. Deutsch.  Best approximation in inner product spaces , 2001 .

[33]  Scott B. Lindstrom,et al.  SURVEY: SIXTY YEARS OF DOUGLAS–RACHFORD , 2018, Journal of the Australian Mathematical Society.

[34]  Joël Benoist,et al.  The Douglas–Rachford algorithm for the case of the sphere and the line , 2015, J. Glob. Optim..

[35]  Heinz H. Bauschke.  New Demiclosedness Principles for (Firmly) Nonexpansive Operators , 2011, 1103.0991.

[36]  Francisco J. Aragón Artacho,et al.  Solving Graph Coloring Problems with the Douglas-Rachford Algorithm , 2016, Set-Valued and Variational Analysis.

[37]  A. Pazy.  Asymptotic behavior of contractions in hilbert space , 1971 .

[38]  Guoyin Li,et al.  Douglas–Rachford splitting for nonconvex optimization with application to nonconvex feasibility problems , 2014, Math. Program..

[39]  S. Banach.  Sur les opérations dans les ensembles abstraits et leur application aux équations intégrales , 1922 .

[40]  Heinz H. Bauschke,et al.  On the asymptotic behaviour of the Aragón Artacho-Campoy algorithm , 2018, Oper. Res. Lett..

[41]  Veit Elser,et al.  The Complexity of Bit Retrieval , 2016, IEEE Transactions on Information Theory.

[42]  Jonathan M. Borwein,et al.  Norm convergence of realistic projection and reflection methods , 2013, 1312.7323.

[43]  Hung M. Phan,et al.  Linear convergence of the generalized Douglas–Rachford algorithm for feasibility problems , 2017, J. Glob. Optim..

[44]  Jonathan M. Borwein,et al.  A Cyclic Douglas–Rachford Iteration Scheme , 2013, J. Optim. Theory Appl..

[45]  Hein Hundal.  An alternating projection that does not converge in norm , 2004 .

[46]  Heinz H. Bauschke,et al.  Linear and strong convergence of algorithms involving averaged nonexpansive operators , 2014, Journal of Mathematical Analysis and Applications.

[47]  Jonathan M. Borwein,et al.  Dynamics of the Douglas-Rachford Method for Ellipses and p-Spheres , 2016, 1610.03975.

[48]  D. Russell Luke,et al.  Nonconvex Notions of Regularity and Convergence of Fundamental Algorithms for Feasibility Problems , 2012, SIAM J. Optim..

[49]  Matthew K. Tam,et al.  A feasibility approach for constructing combinatorial designs of circulant type , 2017, J. Comb. Optim..

[50]  Jason Schaad.  Modeling the 8-queens problem and Sudoku using an algorithm based on projections onto nonconvex sets , 2010 .

[51]  Z. Opial.  Weak convergence of the sequence of successive approximations for nonexpansive mappings , 1967 .

[52]  W. Cheney,et al.  Proximity maps for convex sets , 1959 .

[53]  V Elser,et al.  Searching with iterated maps , 2007, Proceedings of the National Academy of Sciences.

[54]  Jonathan M. Borwein,et al.  Global behavior of the Douglas–Rachford method for a nonconvex feasibility problem , 2015, J. Glob. Optim..

[55]  Jonathan M. Borwein,et al.  Global convergence of a non-convex Douglas–Rachford iteration , 2012, J. Glob. Optim..

[56]  Yair Censor,et al.  New Douglas-Rachford Algorithmic Structures and Their Convergence Analyses , 2015, SIAM J. Optim..

[57]  Karin Schwab,et al.  Best Approximation In Inner Product Spaces , 2016 .

[58]  Heinz H. Bauschke,et al.  On the Finite Convergence of the Douglas-Rachford Algorithm for Solving (Not Necessarily Convex) Feasibility Problems in Euclidean Spaces , 2017, SIAM J. Optim..

[59]  Heinz H. Bauschke,et al.  Finding best approximation pairs relative to two closed convex sets in Hilbert spaces , 2004, J. Approx. Theory.

[60]  H. H. Rachford,et al.  On the numerical solution of heat conduction problems in two and three space variables , 1956 .

[61]  D. Russell Luke,et al.  Alternating Projections and Douglas-Rachford for Sparse Affine Feasibility , 2013, IEEE Transactions on Signal Processing.

[62]  Heinz H. Bauschke,et al.  Affine Nonexpansive Operators, Attouch–Théra Duality and the Douglas–Rachford Algorithm , 2016, 1603.09418.

[63]  Matthew K. Tam,et al.  A Lyapunov-type approach to convergence of the Douglas–Rachford algorithm for a nonconvex setting , 2017, J. Glob. Optim..