Enhanced sparse representation classifier for text classification

Abstract Classification of text based on its substance is an essential part of analysis to organize enormously large text data and to mine the salient information contained in it. It is gaining greater attention with the surge in the volume of on-line data available. Classical algorithms like k-NN (k-nearest neighbor), SVM (Support Vector Machine) and their variations have been observed to yield only reasonable results in addressing the problem, leaving enough room for further improvement. A class of algorithms commonly referred to as Sparse Methods has been emerged recently from compressive sensing and found numerous effective applications in many areas of data analysis and image processing. Sparse Methods as a tool for text analysis is an alley that is largely unexplored rigorously. This paper presents exploration of sparse representation-based methods for text classification. Based on the success of sparse representation based methods in different areas of data analysis, we intuitively hypothesized that it should work well on text classification problems as well. This paper empirically reinforces the hypothesis by testing the method on Reuters and WebKB data sets. The empirical results on Reuters and WebKB benchmark data show that it can outperform classical classification algorithms like SVM and k-NN. It has been observed that obtaining the basis of representation and sparse codes are computationally costly operations affecting the performance of the system. We also propose a class-wise dictionary refinement algorithm and dynamic dictionary selection algorithm to make sparse coding faster. The addition of dictionary refinement to the classification system not only reduces the time taken for sparse coding but also gives improved classification accuracy. The outcomes of the study are empirical verification of sparse representation classifier as a text classification tool and a computationally efficient solution for the bottleneck operation of sparse coding.

[1]  Kichun Lee,et al.  Opinion mining using ensemble text hidden Markov models for text classification , 2018, Expert Syst. Appl..

[2]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[3]  Dirk Thorleuchter,et al.  Technology classification with latent semantic indexing , 2013, Expert Syst. Appl..

[4]  J. Franklin,et al.  The elements of statistical learning: data mining, inference and prediction , 2005 .

[5]  David G. Stork,et al.  Pattern Classification , 1973 .

[6]  Jun Zhao,et al.  Recurrent Convolutional Neural Networks for Text Classification , 2015, AAAI.

[7]  Guy Tsafnat,et al.  Evaluation of a rule-based method for epidemiological document classification towards the automation of systematic reviews , 2017, J. Biomed. Informatics.

[8]  Guillermo Sapiro,et al.  Sparse Representation for Computer Vision and Pattern Recognition , 2010, Proceedings of the IEEE.

[9]  Shengyi Jiang,et al.  An improved K-nearest-neighbor algorithm for text categorization , 2012, Expert Syst. Appl..

[10]  Michael Elad,et al.  A generalized uncertainty principle and sparse representation in pairs of bases , 2002, IEEE Trans. Inf. Theory.

[11]  S. Sathiya Keerthi,et al.  Large scale semi-supervised linear SVMs , 2006, SIGIR.

[12]  Sholom M. Weiss,et al.  Automated learning of decision rules for text categorization , 1994, TOIS.

[13]  Wai Lam,et al.  Using a generalized instance set for automatic text categorization , 1998, SIGIR '98.

[14]  Saso Dzeroski,et al.  Decision trees for hierarchical multi-label classification , 2008, Machine Learning.

[15]  Eibe Frank,et al.  Naive Bayes for Text Classification with Unbalanced Classes , 2006, PKDD.

[16]  Mika V. Mäntylä,et al.  The evolution of sentiment analysis - A review of research topics, venues, and top cited papers , 2016, Comput. Sci. Rev..

[17]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[18]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[19]  Hae-Chang Rim,et al.  Some Effective Techniques for Naive Bayes Text Classification , 2006, IEEE Transactions on Knowledge and Data Engineering.

[20]  Aytug Onan,et al.  Ensemble of keyword extraction methods and classifiers in text classification , 2016, Expert Syst. Appl..

[21]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Honggang Zhang,et al.  Local Sparse Representation Based Classification , 2010, 2010 20th International Conference on Pattern Recognition.

[23]  V. K. Govindan,et al.  Optimizing visual dictionaries for effective image retrieval , 2015, International Journal of Multimedia Information Retrieval.

[24]  Michael Elad,et al.  Linearized Kernel Dictionary Learning , 2015, IEEE Journal of Selected Topics in Signal Processing.

[25]  Shuigeng Zhou,et al.  Effectively classifying short texts by structured sparse representation with dictionary filtering , 2015, Inf. Sci..

[26]  Frédéric Lesage,et al.  The Application of Compressed Sensing for Photo-Acoustic Tomography , 2009, IEEE Transactions on Medical Imaging.

[27]  Andrew McCallum,et al.  A comparison of event models for naive bayes text classification , 1998, AAAI 1998.

[28]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[29]  J. J. Rocchio,et al.  Relevance feedback in information retrieval , 1971 .

[30]  Jian Sun,et al.  Sparse-Coded Features for Image Retrieval , 2013, BMVC.

[31]  David R. Karger,et al.  Tackling the Poor Assumptions of Naive Bayes Text Classifiers , 2003, ICML.

[32]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[33]  Pankoo Kim,et al.  Text analysis for detecting terrorism-related articles on the web , 2014, J. Netw. Comput. Appl..

[34]  Neeraj Sharma,et al.  Text classification using combined sparse representation classifiers and support vector machines , 2016, 2016 4th International Symposium on Computational and Business Intelligence (ISCBI).

[35]  Zhi-Hua Zhou,et al.  Multilabel Neural Networks with Applications to Functional Genomics and Text Categorization , 2006, IEEE Transactions on Knowledge and Data Engineering.

[36]  Brian D. Davison,et al.  Web page classification: Features and algorithms , 2009, CSUR.

[37]  Dimitris Spathis,et al.  A comparison between semi-supervised and supervised text mining techniques on detecting irony in greek political tweets , 2016, Eng. Appl. Artif. Intell..

[38]  Thorsten Joachims.  A Statistical Learning Model of Text Classification for SVMs , 2002 .

[39]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[40]  Thorsten Joachims,et al.  Text Categorization with Support Vector Machines: Learning with Many Relevant Features , 1998, ECML.

[41]  Jean-Luc Starck,et al.  Sparse Solution of Underdetermined Systems of Linear Equations by Stagewise Orthogonal Matching Pursuit , 2012, IEEE Transactions on Information Theory.

[42]  Songbo Tan,et al.  Neighbor-weighted K-nearest neighbor for unbalanced text corpus , 2005, Expert Syst. Appl..

[43]  Mohammad Soleymani,et al.  A survey of multimodal sentiment analysis , 2017, Image Vis. Comput..

[44]  Lina Yao,et al.  Collaborative text categorization via exploiting sparse coefficients , 2018, World Wide Web.

[45]  V. K. Govindan,et al.  A Hybrid Deep Learning Architecture for Latent Topic-based Image Retrieval , 2018, Data Science and Engineering.

[46]  David D. Lewis,et al.  Naive (Bayes) at Forty: The Independence Assumption in Information Retrieval , 1998, ECML.

[47]  Adriano Lorena Inácio de Oliveira,et al.  Smoothed n-gram based models for tweet language identification: A case study of the Brazilian and European Portuguese national varieties , 2017, Appl. Soft Comput..

[48]  Erik Cambria,et al.  Ensemble application of convolutional neural networks and multiple kernel learning for multimodal sentiment analysis , 2017, Neurocomputing.

[49]  Aytug Onan,et al.  A multiobjective weighted voting ensemble classifier based on differential evolution algorithm for text sentiment classification , 2016, Expert Syst. Appl..

[50]  Charu C. Aggarwal,et al.  A Survey of Text Classification Algorithms , 2012, Mining Text Data.

[51]  David Zhang,et al.  A Survey of Sparse Representation: Algorithms and Applications , 2015, IEEE Access.

[52]  Tara N. Sainath,et al.  Sparse representations for text categorization , 2010, INTERSPEECH.

[53]  M. F. Porter,et al.  An algorithm for suffix stripping , 1997 .

[54]  Marcin Mironczuk,et al.  A recent overview of the state-of-the-art elements of text classification , 2018, Expert Syst. Appl..

[55]  Bo Yu,et al.  Latent semantic analysis for text categorization using neural network , 2008, Knowl. Based Syst..

[56]  Cagatay CATAL,et al.  A sentiment classification model based on multiple classifiers , 2017, Appl. Soft Comput..

[57]  Walmir M. Caminhas,et al.  A review of machine learning approaches to Spam filtering , 2009, Expert Syst. Appl..