Arabic text summarization based on latent semantic analysis to enhance arabic documents clustering

Arabic Documents Clustering is an important task for obtaining good results with the traditional Information Retrieval (IR) systems especially with the rapid growth of the number of online documents present in Arabic language. Documents clustering aim to automatically group similar documents in one cluster using different similarity/distance measures. This task is often affected by the documents length, useful information on the documents is often accompanied by a large amount of noise, and therefore it is necessary to eliminate this noise while keeping useful information to boost the performance of Documents clustering. In this paper, we propose to evaluate the impact of text summarization using the Latent Semantic Analysis Model on Arabic Documents Clustering in order to solve problems cited above, using five similarity/distance measures: Euclidean Distance, Cosine Similarity, Jaccard Coefficient, Pearson Correlation Coefficient and Averaged Kullback-Leibler Divergence, for two times: without and with stemming. Our experimental results indicate that our proposed approach effectively solves the problems of noisy information and documents length, and thus significantly improve the clustering performance.

[1]  Marc El-Bèze,et al.  Clustering by means of unsupervised decision trees or hierarchical and K-means-like algorithm , 2000 .

[2]  Richard A. Harshman,et al.  Indexing by Latent Semantic Analysis , 1990, J. Am. Soc. Inf. Sci..

[3]  Leah S. Larkey,et al.  Arabic Information Retrieval at UMass in TREC-10 , 2001, TREC.

[4]  Anna-Lan Huang,et al.  Similarity Measures for Text Document Clustering , 2008 .

[5]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[6]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[7]  George Karypis,et al.  Evaluation of hierarchical clustering algorithms for document datasets , 2002, CIKM '02.

[8]  Lisa Ballesteros,et al.  Improving stemming for Arabic information retrieval: light stemming and co-occurrence analysis , 2002, SIGIR '02.

[9]  Hanane Froud,et al.  A comparative study of root-based and stem-based approaches for measuring the similarity between arabic words for arabic text mining applications , 2012 .

[10]  Riyad Al-Shalabi,et al.  A Computational Morphology System for Arabic , 1998, SEMITIC@COLING.

[11]  S. A. Ouatik,et al.  Stemming and similarity measures for Arabic Documents Clustering , 2010, 2010 5th International Symposium On I/V Communications and Mobile Network.

[12]  T DumaisSusan,et al.  Using linear algebra for intelligent information retrieval , 1995 .

[13]  Wei-Ying Ma,et al.  Web page clustering enhanced by summarization , 2004, CIKM '04.

[14]  Ophir Frieder,et al.  On arabic search: improving the retrieval effectiveness via a light stemming approach , 2002, CIKM '02.

[15]  Naftali Tishby,et al.  The information bottleneck method , 2000, ArXiv.

[16]  Amine Bensaid,et al.  Automatic Arabic Document Categorization Based on the Naïve Bayes Algorithm , 2004 .

[17]  Sameh Ghwanmeh Applying Clustering of Hierarchical K-means-like Algorithm on Arabic Language , 2007 .

[18]  Chinatsu Aone,et al.  Fast and effective text mining using linear-time document clustering , 1999, KDD '99.

[19]  Abdelmonaime Lachkar,et al.  Stemming versus Light Stemming for measuring the simitilarity between Arabic Words with Latent Semantic Analysis model , 2012, 2012 Colloquium in Information Science and Technology.

[20]  Anna M. Gade The Qur'ān : an introduction , 2010 .

[21]  Eric Atwell,et al.  The design of a corpus of Contemporary Arabic , 2006 .

[22]  George Karypis,et al.  Empirical and Theoretical Comparisons of Selected Criterion Functions for Document Clustering , 2004, Machine Learning.

[23]  Susan T. Dumais,et al.  Using Linear Algebra for Intelligent Information Retrieval , 1995, SIAM Rev..

[24]  Fredric C. Gey,et al.  Building an Arabic Stemmer for Information Retrieval , 2002, TREC.

[25]  J. Steinberger,et al.  Using Latent Semantic Analysis in Text Summarization and Summary Evaluation , 2004 .

[26]  Riadh Ouersighni A major offshoot of the DIINAR-MBC project: AraParse, a morpho- syntactic analyzer for unvowelled Arabic texts , 2001 .

[27]  Xin Liu,et al.  Generic text summarization using relevance measure and latent semantic analysis , 2001, SIGIR '01.