23 Automatic Item Generation and Cognitive Psychology

Publisher Summary This chapter discusses item generation and the role of item response theory (IRT) models that permit cognitive variables to predict item parameters. It presents an overview of the methods of item generation and the research requirements for application. It reviews both the item model approach and the cognitive design system approach to item generation. The item model approach has the advantage of being applicable to item generation relatively quickly as it requires a lesser cognitive foundation. The cognitive design approach has the advantages of explicating construct validity at the item level because the level and the source of cognitive complexity in an item are quantified. The chapter also describes psychometric models that are based on IRT. The models reviewed included the linear logistic test model (LLTM), the 2PL-constrained model, and the hierarchical IRT model. The latter has been shown to produce a broad family of models appropriate for item generation with certain constraints applied. The chapter illustrates some estimation procedures for the psychometric models and presents an example of automatic item generation to spatial ability.