Using deep belief networks to extract Chinese entity attribute relation in domain-specific

The state-of-the-art methods used for entity attribute relation extraction are primarily based on statistical machine learning, and the performance strongly depends on the quality of the extracted features. Deep belief networks DBN has been successful in the high dimensional feature space information extraction task, which can without complicated pre-processing. In this paper, the DBN, which consists of one or more restricted Boltzmann machine RBM layers and a back-propagation BP layer, is presented to extract Chinese entity attribute relation in domain-specific. First, the word tokens are transformed to vectors by looking up word embeddings. Then, the RBM layers maintain as much information as possible when feature vectors are transferred to next layer. Finally, the BP layer is trained to classify the features generated by the last RBM layer, and adopting Levenberg-Marquard LM optimisation algorithm to do the training. The experimental results show that the proposed method outperforms state-of-the-art learning models in specific domain entity attribute relation extraction.

[1]  Radford M. Neal Probabilistic Inference Using Markov Chain Monte Carlo Methods , 2011 .

[2]  Tiejun Zhao,et al.  Using Deep Belief Nets for Chinese Named Entity Categorization , 2010, NEWS@ACL.

[3]  Wanxiang Che,et al.  Convolution Neural Network for Relation Extraction , 2013, ADMA.

[4]  Fang Kong,et al.  Exploiting Constituent Dependencies for Tree Kernel-Based Semantic Relation Extraction , 2008, COLING.

[5]  Jun Zhao,et al.  Relation Classification via Convolutional Deep Neural Network , 2014, COLING.

[6]  Geoffrey E. Hinton Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[7]  Nenghai Yu,et al.  Word Alignment Modeling with Context Dependent Deep Neural Network , 2013, ACL.

[8]  Ralph Grishman,et al.  Extracting Relations with Integrated Information Using Kernel Methods , 2005, ACL.

[9]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[10]  Huang Rui-hong Chinese Automatic Entity Relation Extraction , 2007 .

[11]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[12]  David R. Selviah,et al.  Real-Time Nonlinear Parameter Estimation Using the Levenberg–Marquardt Algorithm on Field Programmable Gate Arrays , 2013, IEEE Transactions on Industrial Electronics.

[13]  Chen Yu,et al.  Chinese Relation Extraction Based on Deep Belief Nets , 2012 .

[14]  Marc'Aurelio Ranzato,et al.  Building high-level features using large scale unsupervised learning , 2011, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[15]  Geoffrey E. Hinton,et al.  Application of Deep Belief Networks for Natural Language Understanding , 2014, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[16]  Razvan C. Bunescu,et al.  Subsequence Kernels for Relation Extraction , 2005, NIPS.

[17]  Jian Su,et al.  Exploring Various Knowledge in Relation Extraction , 2005, ACL.

[18]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[19]  Mila Ramos-Santacruz,et al.  REES: A Large-Scale Relation and Event Extraction System , 2000, ANLP.

[20]  Xiaohui Yuan,et al.  Learning from large dataset: segmentation of capsule endoscopy videos , 2012, Int. J. Funct. Informatics Pers. Medicine.

[21]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.