Automated text classification using a dynamic artificial neural network model

Widespread digitization of information in today's internet age has intensified the need for effective textual document classification algorithms. Most real life classification problems, including text classification, genetic classification, medical classification, and others, are complex in nature and are characterized by high dimensionality. Current solution strategies include Naive Bayes (NB), Neural Network (NN), Linear Least Squares Fit (LLSF), k-Nearest-Neighbor (kNN), and Support Vector Machines (SVM); with SVMs showing better results in most cases. In this paper we introduce a new approach called dynamic architecture for artificial neural networks (DAN2) as an alternative for solving textual document classification problems. DAN2 is a scalable algorithm that does not require parameter settings or network architecture configuration. To show DAN2 as an effective and scalable alternative for text classification, we present comparative results for the Reuters-21578 benchmark dataset. Our results show DAN2 to perform very well against the current leading solutions (kNN and SVM) using established classification metrics.

[1]  Yiming Yang,et al.  Using corpus statistics to remove redundant words in text categorization , 1996 .

[2]  Nasser Kehtarnavaz,et al.  DSP-based hierarchical neural network modulation signal classification , 2003, IEEE Trans. Neural Networks.

[3]  Yiming Yang,et al.  Expert network: effective and efficient learning from human decisions in text categorization and retrieval , 1994, SIGIR '94.

[4]  Ricardo Baeza-Yates,et al.  Information Retrieval: Data Structures and Algorithms , 1992 .

[5]  David L. Waltz,et al.  Classifying news stories using memory based reasoning , 1992, SIGIR '92.

[6]  Rajkumar Roy,et al.  TEXT CLASSIFICATION METHOD REVIEW , 2007 .

[7]  Manoochehr Ghiassi,et al.  A dynamic architecture for artificial neural networks , 2005, Neurocomputing.

[8]  Yiming Yang,et al.  A Comparative Study on Feature Selection in Text Categorization , 1997, ICML.

[9]  Manoochehr Ghiassi,et al.  Measuring effectiveness of a dynamic artificial neural network algorithm for classification problems , 2010, Expert Syst. Appl..

[10]  Ken Lang,et al.  NewsWeeder: Learning to Filter Netnews , 1995, ICML.

[11]  Thorsten Joachims,et al.  Training linear SVMs in linear time , 2006, KDD '06.

[12]  David Zimbra,et al.  A dynamic artificial neural network model for forecasting time series events , 2005 .

[13]  Andrew Lumsdaine,et al.  Interconnect agnostic checkpoint/restart in open MPI , 2009, HPDC '09.

[14]  . M.SikanderHayatKhiyal,et al.  Classification of Textual Documents Using Learning Vector Quantization , 2007 .

[15]  Fabrizio Sebastiani,et al.  Machine learning in automated text categorization , 2001, CSUR.

[16]  Yiming Yang,et al.  A re-examination of text categorization methods , 1999, SIGIR '99.

[17]  David D. Lewis,et al.  An evaluation of phrasal and clustered representations on a text categorization task , 1992, SIGIR '92.

[18]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[19]  David L. Waltz,et al.  Trading MIPS and memory for knowledge engineering , 1992, CACM.

[20]  Hinrich Schütze,et al.  Introduction to information retrieval , 2008 .

[21]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[22]  Mehmet Fatih Amasyali,et al.  Cline: A New Decision-Tree Family , 2008, IEEE Transactions on Neural Networks.

[23]  Wei Liu,et al.  Fuzzy HMC Classifiers , 1996, Inf. Sci..

[24]  Takenobu Tokunaga,et al.  Cluster-based text categorization: a comparison of category search strategies , 1995, SIGIR '95.

[25]  Karen Spärck Jones,et al.  Natural language processing for information retrieval , 1996, CACM.

[26]  Yiming Yang,et al.  RCV1: A New Benchmark Collection for Text Categorization Research , 2004, J. Mach. Learn. Res..

[27]  David D. Lewis,et al.  Evaluating and optimizing autonomous text classification systems , 1995, SIGIR '95.

[28]  David D. Lewis,et al.  Representation and Learning in Information Retrieval , 1991 .

[29]  T. Landauer,et al.  Indexing by Latent Semantic Analysis , 1990 .

[30]  Jing Yang,et al.  A parallel SVM training algorithm on large-scale classification problems , 2005, 2005 International Conference on Machine Learning and Cybernetics.

[31]  Philip J. Hayes,et al.  A News Story Categorization System , 1988, ANLP.

[32]  M. E. Maron,et al.  Automatic Indexing: An Experimental Inquiry , 1961, JACM.

[33]  Thorsten Joachims,et al.  Text Categorization with Support Vector Machines: Learning with Many Relevant Features , 1998, ECML.

[34]  Ulrich H.-G. Kreßel,et al.  Pairwise classification and support vector machines , 1999 .

[35]  Gareth J. F. Jones,et al.  Using online linear classifiers to filter spam emails , 2006, Pattern Analysis and Applications.

[36]  Gerald Kowalski,et al.  Information Retrieval Systems: Theory and Implementation , 1997 .

[37]  D. K. Harmon,et al.  Overview of the Third Text Retrieval Conference (TREC-3) , 1996 .

[38]  Jaejoon Lee,et al.  Consensual and Hierarchical Classification of Remotely Sensed Multispectral Images , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[39]  David D. Lewis.  Text representation for intelligent text retrieval: a classification-oriented view , 1992 .

[40]  Thorsten Joachims,et al.  Web Watcher: A Tour Guide for the World Wide Web , 1997, IJCAI.

[41]  Sholom M. Weiss,et al.  Automated learning of decision rules for text categorization , 1994, TOIS.

[42]  Edward Y. Chang,et al.  Parallelizing Support Vector Machines on Distributed Computers , 2007, NIPS.

[43]  Yair Altman,et al.  What Is Java , 2011 .

[44]  Donna K. Harman,et al.  Overview of the Second Text REtrieval Conference (TREC-2) , 1994, HLT.

[45]  William N. Venables,et al.  An Introduction To R , 2004 .

[46]  Hinrich Schütze,et al.  A comparison of classifiers and document representations for the routing problem , 1995, SIGIR '95.

[47]  William R. Hersh,et al.  Evaluation of SAPHIRE: an automated approach to indexing and retrieving medical literature. , 1991, Proceedings. Symposium on Computer Applications in Medical Care.

[48]  Tshilidzi Marwala,et al.  Image Classification Using SVMs: One-against-One Vs One-against-All , 2007, ArXiv.

[49]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[50]  Gerard Salton,et al.  Term-Weighting Approaches in Automatic Text Retrieval , 1988, Inf. Process. Manag..

[51]  David D. Lewis,et al.  Reuters-21578 Text Categorization Test Collection, Distribution 1.0 , 1997 .