Creating Content for Educational Testing Using a Workflow That Supports Automatic Item Generation

Automatic item generation is a rapidly evolving research area where cognitive theories, computer technologies, and psychometric practices are used to create models that produce test items with the aid of computer technology. The purpose of our study is to describe the workflow in a strategic partnership between researchers as the University of Alberta and content specialists at the testing company ACT Inc. In this workflow, technical automated item and content generation expertise was combined with item development and subject-matter expertise for the purpose of producing large numbers of high-quality, content-specific test items. The methods and processes described in our study will also be used to help transform item and passage development at ACT Inc. from what is currently a manual, labor intensive, non-scalable process to a specification driven, automated, highly scalable process.

[1]  Mark J. Gierl,et al.  Developing a Taxonomy of Item Model Types to Promote Assessment Engineering , 2008 .

[2]  James W Pellegrino,et al.  Technology and Testing , 2009, Science.

[3]  Isaac I. Bejar,et al.  A FEASIBILITY STUDY OF ON‐THE‐FLY ITEM GENERATION IN ADAPTIVE TESTING , 2002 .

[4]  Mark J. Gierl,et al.  Using automatic item generation to create multiple‐choice test items , 2012, Medical education.

[5]  Lawrence M. Rudner,et al.  Implementing the Graduate Management Admission Test Computerized Adaptive Test , 2009 .

[6]  Donovan R. Hare,et al.  Assembling an inventory of multistage adaptive testing systems , 2009 .

[7]  Mark J. Gierl,et al.  A Process for Reviewing and Evaluating Generated Test Items. , 2016 .

[8]  A. Laduca,et al.  Item modelling procedure for constructing content‐equivalent multiple choice questions , 1986, Medical education.

[9]  Mark J. Gierl,et al.  A Method for Generating Educational Test Items that are Aligned to the Common Core State Standards , 2015 .

[10]  Mark J. Gierl,et al.  Instructional Topics in Educational Measurement (ITEMS) Module: Using Automated Processes to Generate Test Items , 2013 .

[11]  Mark J. Gierl,et al.  Using Automated Processes to Generate Test Items And Their Associated Solutions and Rationales to Support Formative Feedback , 2015, IxD&A.

[12]  Mark J. Gierl,et al.  Using Automatic Item Generation to Improve the Quality of MCQ Distractors , 2016, Teaching and learning in medicine.

[13]  Mark J. Gierl,et al.  Automatic item generation : theory and practice , 2012 .