Combining unsupervised and invigilated assessment of introductory programming

We compared student performance on large-scale take-home assignments and small-scale invigilated tests that require competency with exactly the same programming concepts. The purpose of the tests, which were carried out soon after the take home assignments were submitted, was to validate the students' assignments as individual work. We found widespread discrepancies between the marks achieved by students between the two types of tasks. Many students were able to achieve a much higher grade on the take-home assignments than the invigilated tests. We conclude that these paired assessments are an effective way to quickly identify students who are still struggling with programming concepts that we might otherwise assume they understand, given their ability to complete similar, yet more complicated, tasks in their own time. We classify these students as not yet being at the neo-Piagetian stage of concrete operational reasoning.

[1]  Jens Bennedsen,et al.  Failure rates in introductory programming , 2007, SGCS.

[2]  Juha Sorva,et al.  Notional machines and introductory programming education , 2013, TOCE.

[3]  Raymond Lister,et al.  A qualitative think aloud study of the early neo-piagetian stages of reasoning in novice programmers , 2013, ACE '13.

[4]  Donna Teague,et al.  Engaging students in programming , 2010, ACE '10.

[5]  John A. Clarke,et al.  Expectations and realities for first year students at an Australian university , 2008 .

[6]  Raymond Lister,et al.  Concrete and other neo-Piagetian forms of reasoning in the novice programmer , 2011, ACE 2011.

[7]  Malcolm W. Corney,et al.  Is anybody there? Bootstrapping attendance with engagement , 2011 .

[8]  Frederick W. B. Li,et al.  Failure rates in introductory programming revisited , 2014, ITiCSE '14.

[9]  Andrew A. Adams,et al.  Student assessment in the ubiquitously connected world , 2011, CSOC.

[10]  Madonna Margaret Teague,et al.  Pedagogy of introductory computer programming : a people-first approach , 2011 .

[11]  W. Hanfer,et al.  Work in progress - authenticating authorship of student work: beyond plagiarism detection , 2005, Proceedings Frontiers in Education 35th Annual Conference.

[12]  Karina Assiter.  Work in progress — Programming assignment summary for analysis, qualitative assessment and continuous improvement in CS1 — CS3 , 2010, 2010 IEEE Frontiers in Education Conference (FIE).

[13]  Raymond Lister,et al.  Mired in the Web: Vignettes from Charlotte and Other Novice Programmers , 2015, ACE.

[14]  António José Mendes,et al.  A study on student performance in first year CS courses , 2010, ITiCSE '10.

[15]  Raymond Lister,et al.  Using neo-piagetian theory, formative in-class tests and think alouds to better understand student thinking: A preliminary report on computer programming , 2012 .

[16]  D. Feldman,et al.  Piaget's stages: the unfinished symphony of cognitive development , 2004 .

[17]  Sally M. Kift,et al.  The next, great first year challenge: Sustaining, coordinating and embedding coherent institution–wide approaches to enact the FYE as "everybody’s business" , 2008 .

[18]  Hisham M. Haddad,et al.  Towards a model of student success in programming courses , 2005, ACM-SE 43.

[19]  Jan Boom.  Commentary on: Piaget's stages: the unfinished symphony of cognitive development , 2004 .

[20]  Judy McKay,et al.  Seven factors that influence ICT student achievement , 2007, ITiCSE '07.

[21]  Allison Elliott Tew,et al.  A fresh look at novice programmers' performance and their teachers' expectations , 2013, ITiCSE -WGR '13.

[22]  Raymond Lister,et al.  Longitudinal think aloud study of a novice programmer , 2014, ACE.

[23]  Tim O'Shea,et al.  The black box inside the glass box: presenting computing concepts to novices , 1999, Int. J. Hum. Comput. Stud..