Pursuit Algorithms – Practice

It is now time to consider reliable and effcient methods for solving (P 0), as a straightforward approach seems hopeless.We now discuss methods which, it seems, have no hope of working – but which, under specific conditions, will work. Looking at the problem (P 0), $${\left(P_o\right):\quad \min\limits_X \parallel \mathbf{X}\parallel_0 \,{\rm subject\,\, to} \mathbf \quad \mathbf{b}=\mathbf{A\mathbf{x}}},$$ one observes that the unknown X is composed of two effective parts to be found – the support of the solution, and the non-zero values over this support. Thus, one way to attack the numerical solution of (P 0) is to focus on the support, with the understanding that once found, the non-zero values of X are easily detected by plain Least-Squares. As the support is discrete in nature, algorithms that seek it are discrete as well. This line of reasoning leads to the family of greedy algorithms that will be presented hereafter.