Structured sparsity: theorems, algorithms and applications

Today, sparsity techniques have been widely used to address practical problems in the fields of medical imaging, machine learning, computer vision, data mining, compressive sensing, image processing, video analysis and multimedia. We will briefly introduce the related sparsity techniques and their successful applications on compressive sensing, sparse learning, computer vision and medical imaging. Then, we propose a new concept called strong group sparsity to develop a theory for group Lasso, which shows that group Lasso is superior to standard Lasso for strongly group-sparse data. It provides a convincing theoretical justification for using group sparsity regularization when the underlying group structure is consistent with the data. Moreover, the theory also predicts the limitations of the group Lasso formulations. To address those limitations, we further build a new framework called structured sparsity, which is a natural extension of the standard sparsity concept in statistical learning and compressive sensing. By allowing arbitrary structures on the feature set or data, this concept generalizes the group sparsity idea. A general theory (Group-RIP) is developed for learning with structured sparsity, based on the notion of coding complexity associated with the structure, which guarantees better performance with more structure information other than pure sparsity. The new sparsity techniques under this framework have been successfully applied to different applications, such as compressive sensing MR imaging, video background subtraction, object tracking in visual surveillance, dynamic scene registration, automatic image annotation, medical image analysis, fast tag separation in cardiac tMRIs, cervigram image segmentation and so on. The improved experimental results in these applications further validate the nice theoretical guarantees of our new theorems and demonstrate the effectiveness of our new framework in the practical applications involved large scale data.