Evaluation of clustering algorithms for word sense disambiguation

Word sense disambiguation in text is still a difficult problem as the best supervised methods require laborious and costly preparation of training data. This work focuses on evaluation of a few selected clustering algorithms in the task of word sense disambiguation. We used five datasets for two languages (English and Polish). Five clustering algorithms (k-means, k-medoids, hierarchical agglomerative clustering, hierarchical divisive clustering, graph-partitioning-based clustering) and two weighting schemes were tested. The best parameters of the algorithms were chosen using 5 × 2 cross validation. BCubed measure was employed for evaluation of clustering. We conclude that with these settings agglomerative hierarchical clustering achieves best results for all the datasets.

[1]  Mitchell P. Marcus,et al.  OntoNotes: The 90% Solution , 2006, NAACL.

[2]  Satanjeev Banerjee,et al.  The Design, Implementation, and Use of the Ngram Statistics Package , 2003, CICLing.

[3]  George Karypis,et al.  Hierarchical Clustering Algorithms for Document Datasets , 2005, Data Mining and Knowledge Discovery.

[4]  Patrick Pantel,et al.  Clustering by committee , 2003 .

[5]  Eneko Agirre,et al.  Word Sense Disambiguation: Algorithms and Applications (Text, Speech and Language Technology) , 2006 .

[6]  Eneko Agirre,et al.  Semeval-2007 Task 2 : Evaluating Word Sense Induction and Discrimination , 2007 .

[7]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[8]  Izabella Thomas.  Maciej PIASECKI, Stanis?aw SZPAKOWICZ, Bartosz BRODA, « A Wordnet from the Ground Up », Oficyna Wydawnicza Politechniki Wroc?awskiej , 2010 .

[9]  Ron Artstein,et al.  Survey Article: Inter-Coder Agreement for Computational Linguistics , 2008, CL.

[10]  Jiawei Han,et al.  Efficient and Effective Clustering Methods for Spatial Data Mining , 1994, VLDB.

[11]  Cristian Grozea,et al.  Finding optimal parameter settings for high performance word sense disambiguation , 2004 .

[12]  Adam Kilgarriff,et al.  The Senseval-3 English lexical sample task , 2004, SENSEVAL@ACL.

[13]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .

[14]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[15]  Adam Kilgarriff,et al.  Framework and Results for English SENSEVAL , 2000, Comput. Humanit..

[16]  Rada Mihalcea,et al.  Building a Sense Tagged Corpus with Open Mind Word Expert , 2002, SENSEVAL.

[17]  Julia Hirschberg,et al.  V-Measure: A Conditional Entropy-Based External Cluster Evaluation Measure , 2007, EMNLP.

[18]  Ted Pedersen.  Computational Approaches to Measuring the Similarity of Short Contexts : A Review of Applications and Methods , 2008, ArXiv.

[19]  Marina Meila,et al.  Comparing Clusterings by the Variation of Information , 2003, COLT.

[20]  Maciej Piasecki,et al.  Evaluating LexCSD-a weakly-supervised method on improved semanticallyannotated corpus in a large scale experiment. , 2010 .

[21]  Stefan Evert,et al.  The Statistics of Word Cooccur-rences: Word Pairs and Collocations , 2004 .

[22]  Adam Przepiórkowski,et al.  The WSD Development Environment , 2009, LTC.

[23]  Julio Gonzalo,et al.  A comparison of extrinsic clustering evaluation metrics based on formal constraints , 2008, Information Retrieval.

[24]  Thomas G. Dietterich.  Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms , 1998, Neural Computation.

[25]  Patrick Pantel,et al.  Discovering word senses from text , 2002, KDD.

[26]  Roberto Navigli,et al.  Word sense disambiguation: A survey , 2009, CSUR.

[27]  Rada Mihalcea,et al.  The Role of Non-Ambiguous Words in Natural Language Disambiguation , 2003 .

[28]  Philip Edmonds.  SENSEVAL: The evaluation of word sense disambiguation systems , 2002 .

[29]  Bartosz Broda,et al.  Evaluation of clustering algorithms for Polish Word Sense Disambiguation , 2010, Proceedings of the International Multiconference on Computer Science and Information Technology.

[30]  George Karypis,et al.  Empirical and Theoretical Comparisons of Selected Criterion Functions for Document Clustering , 2004, Machine Learning.

[31]  Martha Palmer,et al.  SemEval-2007 Task-17: English Lexical Sample, SRL and All Words , 2007, Fourth International Workshop on Semantic Evaluations (SemEval-2007).

[32]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[33]  Maciej Piasecki,et al.  Semi-supervised word sense disambiguation based on weakly controlled sense induction , 2009, 2009 International Multiconference on Computer Science and Information Technology.

[34]  Joydeep Ghosh,et al.  Relationship-Based Clustering and Visualization for High-Dimensional Data Mining , 2003, INFORMS J. Comput..

[35]  George Karypis,et al.  CLUTO - A Clustering Toolkit , 2002 .

[36]  Suresh Manandhar,et al.  SemEval-2010 Task 14: Evaluation Setting for Word Sense Induction & Disambiguation Systems , 2009, SEW@NAACL-HLT.

[37]  Eneko Agirre,et al.  Evaluating and optimizing the parameters of an unsupervised graph-based WSD algorithm , 2006 .