Parameter estimation under uncertainty with Simulated Annealing applied to an ant colony based probabilistic WSD algorithm

In this article we propose a method based on simulated annealing for the parameter estimation of probabilistic algorithms, where the solution provided by the algorithm can vary from execution to execution. Such algorithms are often very interesting to solve complex combinatorial problems, yet they involve many parameters that can be difficult to estimate manually due to their randomized output. We applied and evaluated a method for the parameter estimation of such algorithms and applied it for an Ant Colony Algorithm for WSD. For the evaluation, we used the Semeval 2007 Task 7 corpus. We split the corpus and took in turn one text as a training corpus and the four remaining texts as a test corpus. We tuned the parameters with an increasing number of sentences from the training text in order to estimate the quantity of data necessary to obtain an efficient and general set of parameters. We found that the results greatly depend on the nature of the text, even a very small amount of training sentences can lead to good results if the text has the right properties. RESUME (French) Estimation de parametres a base de Recuit Simule sous incertitude appliquee a un algorithmes a colonies de fourmis probabiliste. Nous proposons une methode basee sur un Recuit Simule pour l’estimation de parametres pour des algorithmes probabilistes ou les solutions generees varient. Ces algorithmes sont souvent tres interessant pour la resolution de problemes combinatoires complexes, mais ils requierent de nombreux parametres pouvant etre difficiles a estimer manuellement a cause de la nature aleatoire des solutions. Nous avons appliquesPlus specifiquement, nous appliquons et evaluons cette methode a pour estimer les parametres de tels algorimes et l’appliquons a un Algorithme a Colonies de Fourmis pour la desambiguisation lexicale. Pour l’evaluation, nous avons utilise le corpus de Semeval 2007 Tâche 7. Nous avons repectivement separe un texte comme corpus d’entrainement et les quatre autres comme corpus de test. Nous estimons les parametres pour un nombre croissant de phrases pour determiner combien de donnees sont necessaires pour obtenir un ensemble de valeurs de parametres generales et efficaces. Nous concluons que la qualite des resultats depends de la nature des textes. Meme des petites quantites de de phrases peuvent suffir a obtenir de bons resultats, du moment que le texte a les bonnes proprietes.

[1]  H. Levene.  Robust tests for equality of variances , 1961 .

[2]  Walter Daelemans,et al.  Combined Optimization of Feature Selection and Algorithm Parameters in Machine Learning of Language , 2003, ECML.

[3]  Venkat Venkatasubramanian,et al.  A hybrid genetic algorithm for efficient parameter estimation of large kinetic models , 2004, Comput. Chem. Eng..

[4]  S. Kay.  Fundamentals of statistical signal processing: estimation theory , 1993 .

[5]  Louise Guthrie,et al.  Lexical Disambiguation using Simulated Annealing , 1992, COLING.

[6]  U. Diwekar,et al.  Stochastic annealing for synthesis under uncertainty , 1995 .

[7]  Seon Ki Park,et al.  Parameter estimation using the genetic algorithm and its impact on quantitative precipitation forecast , 2006 .

[8]  Athanasios Kehagias,et al.  Genetic Algorithm in Parameter Estimation of Nonlinear Dynamic Systems , 1998, PPSN.

[9]  Didier Schwab,et al.  Ant Colony Algorithm for the Unsupervised Word Sense Disambiguation of Texts: Comparison and Evaluation , 2012, COLING.

[10]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[11]  S. Shapiro,et al.  An Analysis of Variance Test for Normality (Complete Samples) , 1965 .

[12]  H. B. Mann,et al.  On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other , 1947 .

[13]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[14]  Urmila M. Diwekar,et al.  Efficient Combinatorial Optimization under Uncertainty. 1. Algorithmic Development , 2002 .

[15]  M. R. Bolhasani,et al.  Parameter Estimation of Vehicle Handling Model Using Genetic Algorithm , 2002 .

[16]  Walter Daelemans,et al.  GAMBL, genetic algorithm optimization of memory-based WSD , 2004, SENSEVAL@ACL.

[17]  Alexander Gelbukh,et al.  Evolutionary Approach to Natural Language Word Sense Disambiguation through Global Coherence Optimization , 2005 .

[18]  William A. Sethares,et al.  Nonlinear parameter estimation via the genetic algorithm , 1994, IEEE Trans. Signal Process..

[19]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[20]  Éric Gaussier,et al.  A Probabilistic Interpretation of Precision, Recall and F-Score, with Implication for Evaluation , 2005, ECIR.

[21]  Didier Schwab,et al.  A Global Ant Colony Algorithm for Word Sense Disambiguation Based on Semantic Relatedness , 2011, PAAMS.

[22]  L. Ingber.  Adaptive Simulated Annealing (ASA) , 1993 .

[23]  Welch Bl.  THE GENERALIZATION OF ‘STUDENT'S’ PROBLEM WHEN SEVERAL DIFFERENT POPULATION VARLANCES ARE INVOLVED , 1947 .

[24]  Lester Ingber,et al.  Adaptive simulated annealing (ASA): Lessons learned , 2000, ArXiv.

[25]  Y. Zhang,et al.  Parameter Estimation By Genetic Algorithms For Nonlinear Regression , 1995 .

[26]  Nancy Ide,et al.  Word Sense Disambiguation with Very Large Neural Networks Extracted from Machine Readable Dictionaries , 1990, COLING.

[27]  Ken Sharman,et al.  Genetic algorithms for maximum likelihood parameter estimation , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[28]  William H. Press,et al.  Numerical Recipes in FORTRAN - The Art of Scientific Computing, 2nd Edition , 1987 .