A Highly Effective Hybrid Model for Sentence Categorization

Sentence categorization is a task to classify sentences by their types, which is very useful for the analysis of many NLP applications. There exist grammar or syntactic rules to determine types of sentences. And keywords like negation word for negative sentences is an important feature. However, no all sentences have rules to classify. Besides, different types of sentences may contain the same keywords whose meaning may be changed by context. We address the first issue by proposing a hybrid model consisting of Decision Trees and Support Vector Machines. In addition, we design a new feature based on N-gram model. The results of the experiments conducted on the sentence categorization dataset in “Good Ideas of China” Competition 2015 show that (1) our model outperforms baseline methods and all online systems in this competition; (2) the effectiveness of our feature is higher than that of features frequently used in NLP.

[1]  Pankaj Gupta,et al.  Fuzzy COTS Selection for Modular Software Systems Based on Cohesion and Coupling under Multiple Applications Environment , 2012, Int. J. Appl. Evol. Comput..

[2]  Steven L. Salzberg,et al.  Book Review: C4.5: Programs for Machine Learning by J. Ross Quinlan. Morgan Kaufmann Publishers, Inc., 1993 , 1994, Machine Learning.

[3]  David A. Landgrebe,et al.  A survey of decision tree classifier methodology , 1991, IEEE Trans. Syst. Man Cybern..

[4]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[5]  William R. Burrows,et al.  CART Decision-Tree Statistical Analysis and Prediction of Summer Season Maximum Surface Ozone for the Vancouver, Montreal, and Atlantic Regions of Canada , 1995 .

[6]  N. Altman.  An Introduction to Kernel and Nearest-Neighbor Nonparametric Regression , 1992 .

[7]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[8]  Yao Tianfang,et al.  Research on Semantic Orientation Analysis for Topics in Chinese Sentences , 2007 .

[9]  J. Ross Quinlan,et al.  Improved Use of Continuous Attributes in C4.5 , 1996, J. Artif. Intell. Res..

[10]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[11]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[12]  Ron Kohavi,et al.  Scaling Up the Accuracy of Naive-Bayes Classifiers: A Decision-Tree Hybrid , 1996, KDD.

[13]  Lei Zhang,et al.  A Survey of Opinion Mining and Sentiment Analysis , 2012, Mining Text Data.

[14]  Mehdi Khashei,et al.  A novel hybrid classification model of artificial neural networks and multiple linear regression models , 2012, Expert Syst. Appl..

[15]  Ileana Comorovski,et al.  Interrogative phrases and the syntax-semantics interface , 1996 .

[16]  Paul E. Utgoff,et al.  Incremental Induction of Decision Trees , 1989, Machine Learning.

[17]  Paul Rowlett,et al.  On the syntactic derivation of negative sentence adverbials , 1993, Journal of French Language Studies.

[18]  Yang Zhihao.  Text Orientation Identification Based on Semantic Comprehension , 2007 .

[19]  Poonam Gupta,et al.  A Survey of Text Question Answering Techniques , 2012 .

[20]  Michael J. Pazzani,et al.  A hybrid user model for news story classification , 1999 .

[21]  Philip S. Yu,et al.  Top 10 algorithms in data mining , 2007, Knowledge and Information Systems.

[22]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.