Stepping up to integrative questions on CS1 exams

In this paper, we explore the use of sequences of small code writing questions ("concept questions") designed to incrementally evaluate single programming concepts. We report on a study of student performance on a CS1 final examination that included a traditional code-writing question and four intentionally corresponding concept questions. We find that the concept questions are significant predictors of performance on both the corresponding code-writing question and the final exam as a whole. We argue that concept questions provide more accurate formative feedback and simplify marking by reducing the number of variants that must be considered. An analysis of responses categorized by the students' previous programming experience suggests that inexperienced students have the most to gain from the use of concept questions.

[1]  Tony Clear,et al.  An Australasian study of reading and comprehension skills in novice programmers, using the bloom and SOLO taxonomies , 2006 .

[2]  Paul J. Feltovich,et al.  Categorization and Representation of Physics Problems by Experts and Novices , 1981, Cogn. Sci..

[3]  Susan Wiedenbeck,et al.  Self-efficacy and mental models in learning to program , 2004, ITiCSE '04.

[4]  Michael C. Loui,et al.  Setting the Scope of Concept Inventories for Introductory Computing Subjects , 2010, TOCE.

[5]  Anne Venables,et al.  A closer look at tracing, explaining and code writing skills in the novice programmer , 2009, ICER '09.

[6]  Mark Weiser,et al.  A study of programming problem representation in novice and expert programmers , 1981, SIGCPR '81.

[7]  Daniel Zingaro,et al.  Reviewing CS1 exam question content , 2011, SIGCSE '11.

[8]  Daryl J. D'Souza,et al.  Exploring programming assessment instruments: a classification scheme for examination questions , 2011, ICER.

[9]  Raymond Lister,et al.  Relationships between reading, tracing and writing skills in introductory programming , 2008, ICER '08.

[10]  Beth Simon,et al.  Evaluating a new exam question: Parsons problems , 2008, ICER '08.

[11]  Simon,et al.  Explaining program code: giving students the answer helps - but only just , 2011, ICER.

[12]  Carsten Schulte,et al.  What do teachers teach in introductory programming? , 2006, ICER '06.

[13]  Angela Carbone,et al.  Going SOLO to assess novice programmers , 2008, SIGCSE 2008.

[14]  Raymond Lister,et al.  Not seeing the forest for the trees: novice programmers and the SOLO taxonomy , 2006, ITICSE '06.

[15]  Kevin F. Collis,et al.  Evaluating the Quality of Learning: The SOLO Taxonomy , 1977 .