Learning conditional random fields with latent sparse features for acronym expansion finding

The ever increasing usage of acronyms in many kinds of documents, including web pages, is becoming an obstacle for average readers. This paper studies the task of finding expansions in documents for a given set of acronyms. We cast the expansion finding problem as a sequence labeling task and adapt Conditional Random Fields (CRF) to solve it. While adapting CRFs, we enhance the performance from two aspects. First, we introduce nonlinear hidden layers to learn better representations of the input data. Second, we design simple and effective features. We create a hand labeled evaluation data based on Wikipedia.org and web crawling. We evaluate the effectiveness of several algorithms in solving the expansion finding problem. The experimental results demonstrate that the new method achieves performs better than Support Vector Machine and standard Conditional Random Fields.

[1]  Peter D. Turney,et al.  A Supervised Learning Approach to Acronym Identification , 2005, Canadian AI.

[2]  Yalou Huang,et al.  Using SVM to Extract Acronyms from Text , 2006, Soft Comput..

[3]  Honglak Lee,et al.  Sparse deep belief net model for visual area V2 , 2007, NIPS.

[4]  Mitchell P. Marcus,et al.  Text Chunking using Transformation-Based Learning , 1995, VLC@ACL.

[5]  Ben Taskar,et al.  Discriminative Probabilistic Models for Relational Data , 2002, UAI.

[6]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[7]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[8]  Jian Peng,et al.  Conditional Neural Fields , 2009, NIPS.

[9]  Xiaojin Zhu,et al.  Kernel conditional random fields: representation and clique selection , 2004, ICML.

[10]  Kazem Taghva,et al.  Recognizing acronyms and their definitions , 1999, International Journal on Document Analysis and Recognition.

[11]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[12]  Mathieu Roche,et al.  Managing the Acronym/Expansion Identification Process for Text-Mining Applications , 2008, Int. J. Softw. Informatics.

[13]  Russ B. Altman,et al.  Research Paper: Creating an Online Dictionary of Abbreviations from MEDLINE , 2002, J. Am. Medical Informatics Assoc..

[14]  Yi Zhang,et al.  Training Conditional Random Fields Using Transfer Learning for Gesture Recognition , 2010, 2010 IEEE International Conference on Data Mining.

[15]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[16]  Stuart Yeates,et al.  Automatic Extraction of Acronyms from Text , 1999, New Zealand Computer Science Research Students' Conference.