Instructional Design + Knowledge Components: A Systematic Method for Refining Instruction

This paper reports on a systematic method used to improve an existing unit of instruction. The method is distinctive in combining steps of instructional design with "knowledge components" from a cognitively-based framework of learning. Instructional design is used to develop assessment instruments that incorporate information about student misconceptions. The method uses the assessment instruments to evaluate student performance and learning gains, while statistical analysis evaluates the quality of the instruments themselves using measures of difficulty and discrimination. Fine-grain insight into possible improvements is enabled by the knowledge components implicated by the assessment. The method is illustrated and evaluated by applying it to a unit of instruction on collection-based iteration in a computational thinking class. Data gathered during this evaluation highlights a number of opportunities within the unit to refine the instruction.