Alternating Projections

Alternating projections is a very simple algorithm for computing a point in the intersection of some convex sets, using a sequence of projections onto the sets. Like a gradient or subgradient method, alternating projections can be slow, but the method can be useful when we have some efficient method, such as an analytical formula, for carrying out the projections. In these notes, we use only the Euclidean norm, Euclidean distance, and Euclidean projection. Suppose C and D are closed convex sets in R, and let PC and PD denote projection on C and D, respectively. The algorithm starts with any x0 ∈ C, and then alternately projects onto C and D: yk = PD(xk), xk+1 = PC(yk), k = 0, 1, 2, . . . This generates a sequence of points xk ∈ C and yk ∈ D. A basic result, due to Cheney and Goldstein [CG59] is the following. If C ∩D 6= ∅, then the sequences xk and yk both converge to a point x ∗ ∈ C∩D. Roughly speaking, alternating projections finds a point in the intersection of the sets, provided they intersect. Note that we’re not claiming the algorithm produces a point in C ∩D in a finite number of steps. We are claiming that the sequence xk (which lies in C) satisfies dist(xk, D) → 0, and likewise for yk. A simple example is illustrated in figure 1. Alternating projections is also useful when the sets do not intersect. In this case we can prove the following. Assume the distance between C and D is achieved (i.e., there exist points in C and D whose distance is dist(C,D)). Then xk → x ∗ ∈ C, and yk → y ∗ ∈ D, where ‖x−y‖2 = dist(C,D). In other words, alternating projections yields a pair of points in C and D that have minimum distance. In this case, alternating projections also yields (in the limit) a hyperplane that separates C and D. A simple example is illustrated in figure 2. There are many variations on and extensions of the basic alternating projections algorithm. For example, we can find a point in the intersection of k > 2 convex sets, by projecting onto C1, then C2, . . . , then Ck, and then repeating the cycle of k projections. (This is called the sequential or cyclic projection algorithm, instead of alternating projection.) We’ll describe several other extensions below.