The Sequence Prediction Model of Latent Variable Conditional Random Fields

This paper concentrates on the labeling of sequences and introduces a latent model of CRFs. The latent variable conditional random field uses a latent variable encoding scheme to record the latent structure of hidden variables and observation data. Our proposed model is evaluated on three tasks of sequence prediction, including recognition of the name (NER), chunking, and reference parsing. Experimental results indicate a latent variable in the proposed model and provide competitive and solid performance on all three sequence predictions.

[1]  L. Baum,et al.  Statistical Inference for Probabilistic Functions of Finite State Markov Chains , 1966 .

[2]  L. Baum,et al.  An inequality with applications to statistical estimation for probabilistic functions of Markov processes and to a model for ecology , 1967 .

[3]  L. Baum,et al.  Growth transformations for functions on manifolds. , 1968 .

[4]  L. Baum,et al.  An inequality and associated maximization technique in statistical estimation of probabilistic functions of a Markov process , 1972 .

[5]  Adwait Ratnaparkhi,et al.  A Maximum Entropy Model for Part-Of-Speech Tagging , 1996, EMNLP.

[6]  Adam L. Berger,et al.  A Maximum Entropy Approach to Natural Language Processing , 1996, CL.

[7]  Andrew McCallum,et al.  Maximum Entropy Markov Models for Information Extraction and Segmentation , 2000, ICML.

[8]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[9]  Jian Su,et al.  Effective Adaptation of Hidden Markov Model-based Named Entity Recognizer for Biomedical Domain , 2003, BioNLP@ACL.

[10]  William W. Cohen,et al.  Semi-Markov Conditional Random Fields for Information Extraction , 2004, NIPS.

[11]  Yoram Singer,et al.  The Hierarchical Hidden Markov Model: Analysis and Applications , 1998, Machine Learning.

[12]  Jun'ichi Tsujii,et al.  Improving the Scalability of Semi-Markov Conditional Random Fields for Named Entity Recognition , 2006, ACL.

[13]  Hai Zhao,et al.  An Improved Chinese Word Segmentation System with Conditional Random Field , 2006, SIGHAN@COLING/ACL.

[14]  Ben Taskar,et al.  Mixture-of-Parents Maximum Entropy Markov Models , 2007, UAI.

[15]  Dan Klein,et al.  Sparse Multi-Scale Grammars for Discriminative Latent Variable Parsing , 2008, EMNLP.

[16]  Dong Yu,et al.  Using continuous features in the maximum entropy model , 2009, Pattern Recognit. Lett..

[17]  Dan Roth,et al.  Design Challenges and Misconceptions in Named Entity Recognition , 2009, CoNLL.

[18]  Daniel Jurafsky,et al.  Distant supervision for relation extraction without labeled data , 2009, ACL.

[19]  Xiao Sun,et al.  Chinese base phrases chunking based on latent semi-CRF model , 2010, Proceedings of the 6th International Conference on Natural Language Processing and Knowledge Engineering(NLPKE-2010).

[20]  Xiao Sun,et al.  Detecting New Words from Chinese Text Using Latent Semi-CRF Models , 2010, IEICE Trans. Inf. Syst..

[21]  Wee Sun Lee,et al.  Semi-Markov Conditional Random Field with High-Order Features , 2011 .

[22]  Ming-Wei Chang,et al.  To Link or Not to Link? A Study on End-to-End Tweet Entity Linking , 2013, NAACL.

[23]  Nan Ye,et al.  Conditional random field with high-order dependencies for sequence labeling and segmentation , 2014, J. Mach. Learn. Res..

[24]  Wei Xu,et al.  Bidirectional LSTM-CRF Models for Sequence Tagging , 2015, ArXiv.

[25]  Yung-Chun Chang,et al.  Enhancing of chemical compound and drug name recognition using representative tag scheme and fine-grained tokenization , 2015, Journal of Cheminformatics.

[26]  Yijia Liu,et al.  Exploring Segment Representations for Neural Segmentation Models , 2016, IJCAI.

[27]  Eduard H. Hovy,et al.  End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF , 2016, ACL.

[28]  Wei Lu,et al.  Weak Semi-Markov CRFs for Noun Phrase Chunking in Informal Text , 2016, HLT-NAACL.

[29]  Jing Lu,et al.  Joint Inference for Event Coreference Resolution , 2016, COLING.

[30]  Sampo Pyysalo,et al.  Attending to Characters in Neural Sequence Labeling Models , 2016, COLING.

[31]  Hinrich Schütze,et al.  Table Filling Multi-Task Recurrent Neural Network for Joint Entity and Relation Extraction , 2016, COLING.