A Joint Probabilistic Classification Model of Relevant and Irrelevant Sentences in Mathematical Word Problems

Estimating the difficulty level of math word problems is an important task for many educational applications. Identification of relevant and irrelevant sentences in math word problems is an important step for calculating the difficulty levels of such problems. This paper addresses a novel application of text categorization to identify two types of sentences in mathematical word problems, namely relevant and irrelevant sentences. A novel joint probabilistic classification model is proposed to estimate the joint probability of classification decisions for all sentences of a math word problem by utilizing the correlation among all sentences along with the correlation between the question sentence and other sentences, and sentence text. The proposed model is compared with i) a SVM classifier which makes independent classification decisions for individual sentences by only using the sentence text and ii) a novel SVM classifier that considers the correlation between the question sentence and other sentences along with the sentence text. An extensive set of experiments demonstrates the effectiveness of the joint probabilistic classification model for identifying relevant and irrelevant sentences as well as the novel SVM classifier that utilizes the correlation between the question sentence and other sentences. Furthermore, empirical results and analysis show that i) it is highly beneficial not to remove stopwords and ii) utilizing part of speech tagging does not make a significant improvement although it has been shown to be effective for the related task of math word problem type classification.

[1]  T. Minka.  A comparison of numerical optimizers for logistic regression , 2004 .

[2]  Martin F. Porter,et al.  An algorithm for suffix stripping , 1997, Program.

[3]  Stephen I. Brown,et al.  The Art of Problem Posing , 1983 .

[4]  I. Arroyo,et al.  Students in AWE : changing their role from consumers to producers of ITS content , 2003 .

[5]  Thorsten Joachims,et al.  Text Categorization with Support Vector Machines: Learning with Many Relevant Features , 1998, ECML.

[6]  Rwey-Lin Shiah.  The Effects of Computer-Assisted Instruction on the Mathematical Problem Solving of Students With Learning Disabilities , 1994 .

[7]  Jay B. Labov,et al.  Learning and Understanding: Improving Advanced Study of Mathematics and Science in U.S. High Schools, NRC Report , 2002 .

[8]  Luo Si,et al.  A probabilistic graphical model for joint answer ranking in question answering , 2007, SIGIR.

[9]  Carole R. Beal,et al.  Problem Posing in AnimalWatch: An Interactive System for Student-Authored Content , 2008, FLAIRS.

[10]  Luo Si,et al.  Automatic Text Categorization of Mathematical Word Problems , 2009, FLAIRS.

[11]  David J. Spiegelhalter,et al.  Probabilistic Networks and Expert Systems , 1999, Information Science and Statistics.

[12]  E. Corte,et al.  Making sense of word problems , 2000 .

[13]  Luo Si,et al.  Learning to Identify Students' Relevant and Irrelevant Questions in a Micro-blogging Supported Classroom , 2010, Intelligent Tutoring Systems.

[14]  Geoffrey E. Hinton,et al.  Learning and relearning in Boltzmann machines , 1986 .

[15]  L. R. Rasmussen,et al.  In information retrieval: data structures and algorithms , 1992 .

[16]  Tsukasa Hirashima,et al.  Learning by Problem-Posing as Sentence-Integration and Experimental Use , 2007, AIED.

[17]  Thomas E. Scruggs,et al.  The Inclusive Classroom: Strategies for Effective Instruction , 1999 .

[18]  Ricardo Baeza-Yates,et al.  Information Retrieval: Data Structures and Algorithms , 1992 .

[19]  Daniel G. Bobrow,et al.  Natural Language Input for a Computer Problem Solving System , 1964 .

[20]  Beverly Park Woolf,et al.  On-line Tutoring for Math Achievement Testing: A Controlled Evaluation , 2007 .

[21]  Fabrizio Sebastiani,et al.  Machine learning in automated text categorization , 2001, CSUR.

[22]  Yiming Yang,et al.  A re-examination of text categorization methods , 1999, SIGIR '99.

[23]  John R. Anderson,et al.  Authoring Content in the PAT Algebra Tutor , 1998 .

[24]  Stan Matwin,et al.  Feature Engineering for Text Classification , 1999, ICML.

[25]  Thorsten Joachims,et al.  Making large-scale support vector machine learning practical , 1999 .

[26]  Luo Si,et al.  Microblogging in a Classroom: Classifying Students' Relevant and Irrelevant Questions in a Microblogging-Supported Classroom , 2011, IEEE Transactions on Learning Technologies.

[27]  Edie Ronhovde,et al.  Making Sense of Word Problems , 2009 .

[28]  M. F. Porter,et al.  An algorithm for suffix stripping , 1997 .

[29]  Daniela Lucangeli,et al.  The Disturbing Effect of Irrelevant Information on Arithmetic Problem Solving in Inattentive Children , 2002, Developmental neuropsychology.

[30]  L. Siegel,et al.  Short-term memory, working memory, and inhibitory control in children with difficulties in arithmetic problem solving. , 2001, Journal of experimental child psychology.