Developing Assessments to Determine Mastery of Programming Fundamentals

Current CS1 learning outcomes are relatively general, specifying tasks such as designing, implementing, testing and debugging programs that use some fundamental programming constructs. These outcomes impact what we teach, our expectations, and our assessments. Although prior work has demonstrated the utility of single concept assessments, most assessments used in formal examinations combine numerous heterogeneous concepts, resulting in complex and difficult tasks. The exclusive use of these traditional assessments results in two major problems. Firstly, teachers who are administering the assessments find it difficult to quantify what students are struggling with. A program that fails to compile or fails to pass test cases provides little information about what a student can successfully achieve. Secondly, it deprives the students of feedback about what they do know and what they can achieve, which we know is one of the most significant factors that influence student learning. Our traditional compound assessment tasks limit the nature and quality of feedback to teachers and students alike, and may contribute to the perceived difficulty and high drop out rates observed in introductory programming courses. This working group aims to decompose existing CS1 learning outcomes into their component parts, and develop assessment items focused on these individual components. We aim to create exemplar assessments that may be used for formative or summative feedback on student understanding of specific components of programming knowledge. Focusing on independent components of programming increases opportunities for students to demonstrate what they can achieve, and may improve diagnosis of student difficulties. It is likely that this project would be of particular relevance to teachers and researchers interested in mastery learning, or those who wish to demonstrate tight coupling between learning outcomes and assessment tasks. It may also be of relevance to teachers and researchers interested in promoting a more positive mindset in the classroom, focusing on what students do know and what they can achieve during a typical CS1 course.