Deterministic and iterative solutions to subset selection problems

Signal decompositions with overcomplete dictionaries are not unique. We present two new approaches for identifying the sparsest representation of a given signal in terms of a given overcomplete dictionary. The first approach is an algebraic approach that attempts to solve the problem by generating other vectors that span the space of minimum dimension that includes the signal. Unlike other current techniques, including our proposed iterative technique, this algebraic approach is guaranteed to find the sparsest representation of the signal under certain conditions. For example, we can always find the exact solution if the size of the dictionary is close to the size of the space or when the dictionary can be represented by a Vandermonde matrix. Although our technique can work for high signal-to-noise cases, the exact solution is only guaranteed in noise-free cases. Our second approach is iterative and can be applied in cases where the algebraic approach cannot be used. This technique is guaranteed to achieve at least a local minimum of the error function representing the difference between the signal and its sparse representation.