ℓ2-ℓ0 Regularization Path Tracking Algorithms

Sparse signal approximation can be formulated as the mixed L2-L0 minimization problem min_x J(x;lambda)=||y-Ax||_2^2+lambda||x||_0. We propose two heuristic search algorithms to minimize J for a continuum of lambda-values, yielding a sequence of coarse to fine approximations. Continuation Single Best Replacement is a bidirectional greedy algorithm adapted from the Single Best Replacement algorithm previously proposed for minimizing J for fixed lambda. L0 regularization path track is a more complex algorithm exploiting that the L2-L0 regularization path is piecewise constant with respect to lambda. Tracking the L0 regularization path is done in a sub-optimal manner by maintaining (i) a list of subsets that are candidates to be solution supports for decreasing lambda's and (ii) the list of critical lambda-values around which the solution changes. Both algorithms gradually construct the L0 regularization path by performing single replacements, i.e., adding or removing a dictionary atom from a subset. A straightforward adaptation of these algorithms yields sub-optimal solutions to min_x ||y-Ax||_2^2 subject to ||x||_0 =0 and to min_x ||x||_0 subject to ||y-Ax||_2^2<= epsilon for continuous values of epsilon. Numerical simulations show the effectiveness of the algorithms on a difficult sparse deconvolution problem inducing a highly correlated dictionary A.