A Hybrid Engineering Process for Semi-automatic Item Generation

Test authors can generate test items (semi-) automatically with different approaches. On the one hand, bottom-up approaches consist of generating items from sources such as texts or domain models. However, relating the generated items to competence models, which define required knowledge and skills on a proficiency scale remains a challenge. On the other hand, top-down approaches use cognitive models and competence constructs to specify the knowledge and skills to be assessed. Unfortunately, on this high abstraction level it is impossible to identify which item elements can actually be generated automatically. In this paper we present a hybrid process which integrates both approaches. It aims at securing a traceability between the specification levels and making it possible to influence item generation during runtime, i.e., after designing all the intermediate models. In the context of the European project EAGLE, we use this process to generate items for information literacy with a focus on text comprehension.