A Bidirectional Greedy Heuristic for the Subspace Selection Problem

The Subspace Selection Problem (SSP) amounts to selecting t out of n given vectors of dimension m, such that they span a subspace in which a given target b ∈ Rm has a closest possible approximation. This model has numerous applications in e.g. signal compression and statistical regression. It is well known that the problem is NP-hard. Based on elements from a forward and a backward greedy method, we develop a randomized search heuristic, which in some sense resembles variable neighborhood search, for SSP. Through numerical experiments we demonstrate that this approach has good promise, as it produces good results at modest computational cost.

[1]  A. Atkinson.  Subset Selection in Regression , 1992 .

[2]  S. J. Reeves.  An efficient implementation of the backward greedy algorithm for sparse signal reconstruction , 1999, IEEE Signal Processing Letters.

[3]  Yoram Bresler,et al.  On the Optimality of the Backward Greedy Algorithm for the Subset Selection Problem , 2000, SIAM J. Matrix Anal. Appl..

[4]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[5]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[6]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[7]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[8]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[9]  Dag Haugland,et al.  Local search methods for ℓ1-minimization in frame based signal compression , 2006 .

[10]  Pierre Hansen,et al.  Variable Neighborhood Search , 2018, Handbook of Heuristics.

[11]  Gene H. Golub,et al.  Matrix computations , 1983 .

[12]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[13]  Thomas S. Huang,et al.  A fast orthogonal matching pursuit algorithm , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).