Automatic item generation of probability word problems

Abstract Mathematical word problems represent a common item format for assessing student competencies. Automatic item generation (AIG) is an effective way of constructing many items with predictable difficulties, based on a set of predefined task parameters. The current study presents a framework for the automatic generation of probability word problems based on templates that allow for the generation of word problems involving different topics from probability theory. It was tested in a pilot study with N = 146 German university students. The items show a good fit to the Rasch model. Item difficulties can be explained by the Linear Logistic Test Model (LLTM) and by the random-effects LLTM. The practical implications of these findings for future test development in the assessment of probability competencies are also discussed.

[1]  H. Holling,et al.  Explaining and Controlling for the Psychometric Properties of Computer-Generated Figural Matrix Items , 2008 .

[2]  Paul De Boeck,et al.  The Random Weights Linear Logistic Test Model , 2002 .

[3]  Jürgen Rost,et al.  A Conditional Item-Fit Index for Rasch Models , 1994 .

[4]  Georg Gittler,et al.  Automatic Generation of Quantitative Reasoning Items , 2006 .

[5]  D. Cummins.  Children's Interpretations of Arithmetic Word Problems , 1991 .

[6]  David H. Jonassen,et al.  Designing Research-Based Instruction for Story Problems , 2003 .

[7]  Yan Ping Xin,et al.  Word Problem Solving Tasks in Textbooks and Their Relation to Student Performance , 2007 .

[8]  Mitchell J. Nathan,et al.  The Real Story Behind Story Problems: Effects of Representations on Quantitative Reasoning , 2004 .

[9]  Georg Rasch,et al.  Probabilistic Models for Some Intelligence and Attainment Tests , 1981, The SAGE Encyclopedia of Research Design.

[10]  G. H. Fischer,et al.  The linear logistic test model as an instrument in educational research , 1973 .

[11]  M. Meulders,et al.  Cross-Classification Multilevel Logistic Models in Psychometrics , 2003 .

[12]  J. Schepers,et al.  Models with item and item group predictors , 2004 .

[13]  Kathleen M. Sheehan,et al.  ITEMS BY DESIGN: THE IMPACT OF SYSTEMATIC FEATURE VARIATION ON ITEM STATISTICAL CHARACTERISTICS , 1999 .

[14]  P. Boeck,et al.  Explanatory item response models : a generalized linear and nonlinear approach , 2004 .

[15]  M. Sebrechts,et al.  Using Algebra Word Problems to Assess Quantitative Ability: Attributes, Strategies, and Errors , 1996 .