Batch Mode Sparse Active Learning

Sparse representation, due to its clear and powerful insight deep into the structure of data, has seen a recent surge of interest in the classification community. Based on this, a family of reliable classification methods have been proposed. On the other hand, obtaining sufficiently labeled training data has long been a challenging problem, thus considerable research has been done regarding active selection of instances to be labeled. In our work, we will present a novel unified framework, i.e. BMSAL({\em Batch Mode Sparse Active Learning}). Based on the existing sparse family of classifiers, we define rigorously the corresponding BMSAL family and explore their shared properties, most importantly (approximate) sub modularity. We focus on the feasibility and reliability of the BMSAL family: The first one inspires us to optimize the algorithms and conduct experiments comparing with state-of-the-art methods, for reliability, we give error-bounded algorithms, as well as detailed logical deductions and empirical tests for applying sparse in non-linear data sets.

[1]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[2]  Edoardo Amaldi,et al.  On the Approximability of Minimizing Nonzero Variables or Unsatisfied Relations in Linear Systems , 1998, Theor. Comput. Sci..

[3]  Rong Jin,et al.  Large-scale text categorization by batch mode active learning , 2006, WWW '06.

[4]  Guillermo Sapiro,et al.  Discriminative learned dictionaries for local image analysis , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Lihi Zelnik-Manor,et al.  Approximate Nearest Subspace Search with Applications to Pattern Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Ronen Basri,et al.  Lambertian Reflectance and Linear Subspaces , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Piotr Indyk.  Explicit constructions for compressed sensing of sparse signals , 2008, SODA '08.

[8]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[9]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[10]  François-Xavier Le Dimet,et al.  Deblurring From Highly Incomplete Measurements for Remote Sensing , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Trevor Darrell,et al.  Nearest-Neighbor Methods in Learning and Vision , 2008, IEEE Trans. Neural Networks.

[12]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[13]  Allen Y. Yang,et al.  Feature Selection in Face Recognition: A Sparse Representation Perspective , 2007 .

[14]  Rong Jin,et al.  Semisupervised SVM batch mode active learning with applications to image retrieval , 2009, TOIS.

[15]  B. Cranen,et al.  Noise reduction through compressed sensing , 2008, INTERSPEECH.

[16]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[17]  Junfeng Yang,et al.  Alternating Direction Algorithms for 1-Problems in Compressive Sensing , 2009, SIAM J. Sci. Comput..

[18]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  M. L. Fisher,et al.  An analysis of approximations for maximizing submodular set functions—I , 1978, Math. Program..

[20]  B. Cranen,et al.  Noise robust digit recognition using sparse representations , 2008 .

[21]  Emmanuel J. Candès,et al.  Quantitative Robust Uncertainty Principles and Optimally Sparse Decompositions , 2004, Found. Comput. Math..

[22]  E. Candès,et al.  Error correction via linear programming , 2005, FOCS 2005.

[23]  D. Donoho.  For most large underdetermined systems of equations, the minimal 𝓁1‐norm near‐solution approximates the sparsest near‐solution , 2006 .

[24]  Andreas Krause,et al.  Submodular Dictionary Selection for Sparse Representation , 2010, ICML.

[25]  David L. Donoho,et al.  Sparse Solution Of Underdetermined Linear Equations By Stagewise Orthogonal Matching Pursuit , 2006 .

[26]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[27]  J. Lafferty,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[28]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[29]  F. Gianfelici,et al.  Nearest-Neighbor Methods in Learning and Vision (Shakhnarovich, G. et al., Eds.; 2006) [Book review] , 2008 .

[30]  Jeffrey L. Elman,et al.  Analyzing Cross-Connected Networks , 1993, NIPS.

[31]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[32]  John Wright,et al.  Decomposing background topics from keywords by principal component pursuit , 2010, CIKM.

[33]  Bhaskar D. Rao,et al.  Sparse Bayesian learning for basis selection , 2004, IEEE Transactions on Signal Processing.

[34]  Ying Nian Wu,et al.  Wavelet, active basis, and shape script: a tour in the sparse land , 2010, MIR '10.

[35]  Guillermo Sapiro,et al.  Supervised Dictionary Learning , 2008, NIPS.

[36]  Andreas Krause,et al.  Near-Optimal Sensor Placements in Gaussian Processes: Theory, Efficient Algorithms and Empirical Studies , 2008, J. Mach. Learn. Res..