Using Deep Belief Nets for Chinese Named Entity Categorization

Identifying named entities is essential in understanding plain texts. Moreover, the categories of the named entities are indicative of their roles in the texts. In this paper, we propose a novel approach, Deep Belief Nets (DBN), for the Chinese entity mention categorization problem. DBN has very strong representation power and it is able to elaborately self-train for discovering complicated feature combinations. The experiments conducted on the Automatic Context Extraction (ACE) 2004 data set demonstrate the effectiveness of DBN. It outperforms the state-of-the-art learning models such as SVM or BP neural network.

[1]  Yoram Singer,et al.  Unsupervised Models for Named Entity Classification , 1999, EMNLP.

[2]  Hideki Isozaki,et al.  Efficient Support Vector Classifiers for Named Entity Recognition , 2002, COLING.

[3]  S Kullback,et al.  LETTER TO THE EDITOR: THE KULLBACK-LEIBLER DISTANCE , 1987 .

[4]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[5]  David D. McDonald.  Internal and External Evidence in the Identification and Semantic Categorization of Proper Names , 1993 .

[6]  Marie-Francine Moens,et al.  Efficient Hierarchical Entity Classifier Using Conditional Random Fields , 2006, OntologyLearning@COLING/ACL.

[7]  Max Welling Donald,et al.  Products of Experts , 2007 .

[8]  Qin Lu,et al.  Detecting, categorizing and clustering entity mentions in Chinese text , 2007, SIGIR.

[9]  Gang Hu,et al.  Chinese Named Entity Recognition Based on Multilevel Linguistic Features , 2004, IJCNLP.

[10]  Jian Su,et al.  Named Entity Recognition using an HMM-based Chunk Tagger , 2002, ACL.

[11]  Nina Wacholder,et al.  Disambiguation of Proper Names in Text , 1997, ANLP.

[12]  Yoshua Bengio,et al.  Scaling learning algorithms towards AI , 2007 .

[13]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[14]  A. Stuart.  How to get on , 1991, Nature.

[15]  Geoffrey E. Hinton.  Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[16]  Xiaoqiang Luo,et al.  HowtogetaChineseName(Entity): Segmentation and Combination Issues , 2003, EMNLP.