Collaborative text categorization via exploiting sparse coefficients

Text categorization is widely characterized as a multi-label classification problem. Robust modeling of the semantic similarity between a query text and training texts is essential to construct an effective and accurate classifier. In this paper, we systematically investigate the Web page/text classification problem via integrating sparse representation with random measurements. In particular, we first adopt a very sparse data-independent random measurement matrix to map the original high dimensional text feature space to a lower dimensional space without loss of key information. We then propose a generic sparse representation method to obtain the sparse solution by decoding the semantic correlations between the query text and entire training samples. Based on the above method, we also design and examine a series of rules by taking advantage of the sparse coefficients to propagate multiple labels for the given query texts. We have conducted extensive experiments using real-world datasets to examine our proposed approach, and the results show the effectiveness of the proposed approach.

[1]  H. Hotelling Relations Between Two Sets of Variates , 1936 .

[2]  Charu C. Aggarwal,et al.  A Survey of Text Clustering Algorithms , 2012, Mining Text Data.

[3]  Naonori Ueda,et al.  Parametric Mixture Models for Multi-Labeled Text , 2002, NIPS.

[4]  Kenneth Ward Church,et al.  Very sparse random projections , 2006, KDD '06.

[5]  Jieping Ye,et al.  A shared-subspace learning framework for multi-label classification , 2010, TKDD.

[6]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[7]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Foster J. Provost,et al.  Classification in Networked Data: a Toolkit and a Univariate Case Study , 2007, J. Mach. Learn. Res..

[9]  Yaakov Tsaig,et al.  Fast Solution of $\ell _{1}$ -Norm Minimization Problems When the Solution May Be Sparse , 2008, IEEE Transactions on Information Theory.

[10]  Shuicheng Yan,et al.  Learning With $\ell ^{1}$-Graph for Image Analysis , 2010, IEEE Transactions on Image Processing.

[11]  Antonio Torralba,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 80 Million Tiny Images: a Large Dataset for Non-parametric Object and Scene Recognition , 2022 .

[12]  Dimitris Achlioptas,et al.  Database-friendly random projections: Johnson-Lindenstrauss with binary coins , 2003, J. Comput. Syst. Sci..

[13]  Gang Chen,et al.  Semi-supervised Multi-label Learning by Solving a Sylvester Equation , 2008, SDM.

[14]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .

[15]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[16]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[17]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[18]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[19]  Jian Pei,et al.  Consensus-Based Ranking of Multivalued Objects: A Generalized Borda Count Approach , 2014, IEEE Transactions on Knowledge and Data Engineering.

[20]  Jennifer Neville,et al.  Why collective inference improves relational classification , 2004, KDD.

[21]  Lina Yao,et al.  Things of Interest Recommendation by Leveraging Heterogeneous Relations in the Internet of Things , 2016, ACM Trans. Internet Techn..

[22]  Lina Yao,et al.  Multi-label classification via learning a unified object-label graph with sparse representation , 2015, World Wide Web.

[23]  Jieping Ye,et al.  Large-scale sparse logistic regression , 2009, KDD.

[24]  Yi Liu,et al.  Semi-supervised Multi-label Learning by Constrained Non-negative Matrix Factorization , 2006, AAAI.

[25]  Huan Liu,et al.  Relational learning via latent social dimensions , 2009, KDD.

[26]  René Vidal,et al.  Sparse subspace clustering , 2009, CVPR.

[27]  Volker Tresp,et al.  Multi-label informed latent semantic indexing , 2005, SIGIR '05.

[28]  Andrew McCallum,et al.  Collective multi-label classification , 2005, CIKM '05.

[29]  Neeraj Sharma,et al.  Text classification using combined sparse representation classifiers and support vector machines , 2016, 2016 4th International Symposium on Computational and Business Intelligence (ISCBI).

[30]  Shuigeng Zhou,et al.  Effectively classifying short texts by structured sparse representation with dictionary filtering , 2015, Inf. Sci..

[31]  Rui Li,et al.  Exploring social tagging graph for web object classification , 2009, KDD.

[32]  Dale Schuurmans,et al.  Semi-supervised Multi-label Classification - A Simultaneous Large-Margin, Subspace Learning Approach , 2012, ECML/PKDD.

[33]  Brian D. Davison,et al.  Web page classification: Features and algorithms , 2009, CSUR.

[34]  Kun Zhang,et al.  Multi-label learning by exploiting label dependency , 2010, KDD.

[35]  Yihong Gong,et al.  Multi-labelled classification using maximum entropy method , 2005, SIGIR '05.

[36]  Susan T. Dumais,et al.  Partially labeled topic models for interpretable text mining , 2011, KDD.

[37]  Tara N. Sainath,et al.  Sparse representations for text categorization , 2010, INTERSPEECH.

[38]  Michael Elad,et al.  Sparse Representation for Color Image Restoration , 2008, IEEE Transactions on Image Processing.

[39]  Yiming Yang,et al.  An Evaluation of Statistical Approaches to Text Categorization , 1999, Information Retrieval.

[40]  Nicole Immorlica,et al.  Locality-sensitive hashing scheme based on p-stable distributions , 2004, SCG '04.

[41]  Thorsten Joachims,et al.  Text Categorization with Support Vector Machines: Learning with Many Relevant Features , 1998, ECML.

[42]  Jianmin Wang,et al.  Transfer Learning with Graph Co-Regularization , 2012, IEEE Transactions on Knowledge and Data Engineering.

[43]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.