Latent semantics in Named Entity Recognition

Language independent Named Entity Recognition system.Novel features based on latent semantics.Experiments on multiple languages - English, Spanish, Dutch, Czech.State-of-the-art results. In this paper, we propose new features for Named Entity Recognition (NER) based on latent semantics. Furthermore, we explore the effect of unsupervised morphological information on these methods and on the NER system in general. The newly created NER system is fully language-independent thanks to the unsupervised nature of the proposed features. We evaluate the system on English, Spanish, Dutch and Czech corpora and study the difference between weakly and highly inflectional languages. Our system achieves the same or even better results than state-of-the-art language dependent systems. The proposed features proved to be very useful and are the main reason of our promising results.

[1]  Danielle S. McNamara,et al.  Computational Methods to Extract Meaning From Text and Advance Theories of Human Cognition , 2011, Top. Cogn. Sci..

[2]  J. R. Firth,et al.  A Synopsis of Linguistic Theory, 1930-1955 , 1957 .

[3]  Joe F. Zhou,et al.  Proceedings of the 1999 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora, : 21-22 June 1999, University of Maryland, College Park, MD, USA , 1999 .

[4]  John Atkinson,et al.  A multi-strategy approach to biological named entity recognition , 2012, Expert Syst. Appl..

[5]  George Karypis,et al.  CLUTO - A Clustering Toolkit , 2002 .

[6]  Dekang Lin,et al.  Phrase Clustering for Discriminative Learning , 2009, ACL.

[7]  Christopher D. Manning,et al.  Incorporating Non-local Information into Information Extraction Systems by Gibbs Sampling , 2005, ACL.

[8]  Luca Cagliero,et al.  Multi-document summarization based on the Yago ontology , 2013, Expert Syst. Appl..

[9]  Grzegorz Chrupala,et al.  Efficient induction of probabilistic word classes with LDA , 2011, IJCNLP.

[10]  Giovanni Felici,et al.  Soft-constrained inference for Named Entity Recognition , 2014, Inf. Process. Manag..

[11]  Mark Steyvers,et al.  Finding scientific topics , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[12]  Miloslav Konopík,et al.  HPS: High precision stemmer , 2015, Inf. Process. Manag..

[13]  Curt Burgess,et al.  Producing high-dimensional semantic spaces from lexical co-occurrence , 1996 .

[14]  Hideki Isozaki,et al.  Efficient Support Vector Classifiers for Named Entity Recognition , 2002, COLING.

[15]  Miloslav Konopík,et al.  SWSNL: Semantic Web Search Using Natural Language , 2013, Expert Syst. Appl..

[16]  Jason Weston,et al.  A unified architecture for natural language processing: deep neural networks with multitask learning , 2008, ICML '08.

[17]  E. Rosch,et al.  Family resemblances: Studies in the internal structure of categories , 1975, Cognitive Psychology.

[18]  Yoram Singer,et al.  Unsupervised Models for Named Entity Classification , 1999, EMNLP.

[19]  Patrick Pantel,et al.  From Frequency to Meaning: Vector Space Models of Semantics , 2010, J. Artif. Intell. Res..

[20]  Robert L. Mercer,et al.  Class-Based n-gram Models of Natural Language , 1992, CL.

[21]  Miloslav Konopík,et al.  Semantic spaces for improving language modeling , 2014, Comput. Speech Lang..

[22]  Michal Konkol,et al.  CRF-Based Czech Named Entity Recognizer and Consolidation of Czech NER Research , 2013, TSD.

[23]  T. Landauer,et al.  Indexing by Latent Semantic Analysis , 1990 .

[24]  Erik F. Tjong Kim Sang,et al.  Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition , 2003, CoNLL.

[25]  Tom,et al.  Semantic Spaces for Improving Language Modeling I , 2013 .

[26]  Xavier Carreras,et al.  Named Entity Extraction using AdaBoost , 2002, CoNLL.

[27]  Ralph Grishman,et al.  A Maximum Entropy Approach to Named Entity Recognition , 1999 .

[28]  P. Kanerva,et al.  Permutations as a means to encode order in word space , 2008 .

[29]  Magnus Sahlgren,et al.  An Introduction to Random Indexing , 2005 .

[30]  Jan Snajder,et al.  Event graphs for information retrieval and multi-document summarization , 2014, Expert Syst. Appl..

[31]  Erik F. Tjong Kim Sang,et al.  Introduction to the CoNLL-2002 Shared Task: Language-Independent Named Entity Recognition , 2002, CoNLL.

[32]  Dan Roth,et al.  Design Challenges and Misconceptions in Named Entity Recognition , 2009, CoNLL.

[33]  Thomas Hofmann,et al.  Probabilistic Latent Semantic Analysis , 1999, UAI.

[34]  Ted Pedersen,et al.  Word Sense Discrimination by Clustering Contexts in Vector and Similarity Spaces , 2004, CoNLL.

[35]  Naoaki Okazaki,et al.  Named entity recognition with multiple segment representations , 2013, Inf. Process. Manag..

[36]  Jason J. Jung Online named entity recognition method for microtexts in social networking services: A case study of twitter , 2012, Expert Syst. Appl..

[37]  Curt Burgess,et al.  Modelling Parsing Constraints with High-dimensional Context Space , 1997 .

[38]  Zdenek Zabokrtský,et al.  Named Entities in Czech: Annotating Data and Developing NE Tagger , 2007, TSD.

[39]  Asif Ekbal,et al.  A multiobjective simulated annealing approach for classifier ensemble: Named entity recognition in Indian languages as case studies , 2011, Expert Syst. Appl..

[40]  Antonio Toral,et al.  Fine Tuning Features and Post-processing Rules to Improve Named Entity Recognition , 2006, NLDB.

[41]  Behrang Mohit,et al.  Named Entity Recognition , 2014, NLP of Semitic Languages.

[42]  Xavier Carreras,et al.  A Simple Named Entity Extractor using AdaBoost , 2003, CoNLL.

[43]  Zornitsa Kozareva Bootstrapping Named Entity Recognition with Automatically Generated Gazetteer Lists , 2006, EACL.

[44]  Chengqing Zong,et al.  A Joint Model to Identify and Align Bilingual Named Entities , 2013, CL.

[45]  Yoshua Bengio,et al.  Word Representations: A Simple and General Method for Semi-Supervised Learning , 2010, ACL.

[46]  Hang Li,et al.  Named entity recognition in query , 2009, SIGIR.

[47]  John B. Goodenough,et al.  Contextual correlates of synonymy , 1965, CACM.

[48]  W. Charles Contextual correlates of meaning , 2000, Applied Psycholinguistics.

[49]  Michael N. Jones,et al.  Redundancy in Perceptual and Linguistic Experience: Comparing Feature-Based and Distributional Models of Semantic Representation , 2010, Top. Cogn. Sci..

[50]  Geoffrey E. Hinton,et al.  Three new graphical models for statistical language modelling , 2007, ICML '07.

[51]  Jian Su,et al.  Named Entity Recognition using an HMM-based Chunk Tagger , 2002, ACL.

[52]  Maksim Tkatchenko,et al.  Named entity recognition: Exploring features , 2012, KONVENS.

[53]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[54]  Wei Li,et al.  Early results for Named Entity Recognition with Conditional Random Fields, Feature Induction and Web-Enhanced Lexicons , 2003, CoNLL.

[55]  Josef Steinberger,et al.  Multilingual Statistical News Summarization , 2013, Multi-source, Multilingual Information Extraction and Summarization.

[56]  Anselmo Peñas,et al.  Answering questions about European legislation , 2013, Expert Syst. Appl..

[57]  Michael N Jones,et al.  Representing word meaning and order information in a composite holographic lexicon. , 2007, Psychological review.

[58]  Tong Zhang,et al.  Named Entity Recognition through Classifier Combination , 2003, CoNLL.

[59]  James R. Curran,et al.  Language Independent NER using a Maximum Entropy Tagger , 2003, CoNLL.

[60]  Keith Stevens,et al.  The S-Space Package: An Open Source Package for Word Space Models , 2010, ACL.

[61]  Ralph Grishman,et al.  Message Understanding Conference- 6: A Brief History , 1996, COLING.