Mining autograding data in computer science education

In this paper we present an analysis of the impact of instant feedback and autograding in computer science education, beyond the classic Introduction to Programming subject. We analysed the behaviour of 1st year to 4th year students when submitting programming assignments at the University of Sydney over a period of 3 years. These assignments were written in different programming languages, such as C, C++, Java and Python, for diverse computer science courses, from fundamental ones---algorithms, complexity, formal languages, data structures and artificial intelligence to more "practical" ones---programming, distributed systems, databases and networks. We observed that instant feedback and autograding can help students and instructors in subjects not necessarily focused on programming. We also discuss the relationship between the student performance in these subjects and the choice of programming languages or the times at which a student starts and stops working on an assignment.

[1]  V. Shute.  Focus on Formative Feedback , 2008 .

[2]  Anne Venables,et al.  Programming students need instant feedback! , 2003, ACE.

[3]  Tim McGuire.  Introducing multi-core programming into the lower-level curriculum: an incremental approach , 2010 .

[4]  John R. Anderson,et al.  Locus of feedback control in computer-based tutoring: impact on learning rate, achievement and attitudes , 2001, CHI.

[5]  Kirsti Ala-Mutka,et al.  A Survey of Automated Assessment Approaches for Programming Assignments , 2005, Comput. Sci. Educ..

[6]  Julio C. Caiza,et al.  PROGRAMMING ASSIGNMENTS AUTOMATIC GRADING: REVIEW OF TOOLS AND IMPLEMENTATIONS , 2013 .

[7]  Dejan S. Milojicic.  Autograding in the Cloud: Interview with David O'Hallaron , 2011, IEEE Internet Computing.

[8]  Andrew Lim,et al.  On automated grading of programming assignments in an academic institution , 2003, Comput. Educ..

[9]  Salil S. Kanhere,et al.  Impact of an e-learning platform on CSE lectures , 2011, ITiCSE '11.

[10]  David Hovemeyer,et al.  Experiences with marmoset: designing and using an advanced submission and testing system for programming courses , 2006, ITICSE '06.

[11]  Stephen H. Edwards,et al.  Web-CAT: automatically grading programming assignments , 2008, SIGCSE 2008.

[12]  Irena Koprinska,et al.  Predicting Student Performance from Multiple Data Sources , 2015, AIED.

[13]  Ewan D. Tempero,et al.  CodeWrite: supporting student-driven practice of java , 2011, SIGCSE.

[14]  Stephen H. Edwards,et al.  Web-CAT: automatically grading programming assignments , 2008, ITiCSE.

[15]  Fred Martin,et al.  Impact of auto-grading on an introductory computing course , 2013 .

[16]  Sumit Gulwani,et al.  Automated feedback generation for introductory programming assignments , 2012, PLDI.

[17]  Viggo Kann,et al.  Five years with kattis — Using an automated assessment system in teaching , 2011, 2011 Frontiers in Education Conference (FIE).

[18]  MilojicicDejan.  Autograding in the Cloud , 2011 .

[19]  Armando Fox,et al.  From MOOCs to SPOCs , 2013, CACM.

[20]  Vincent Gramoli,et al.  More than you ever wanted to know about synchronization: synchrobench, measuring the impact of the synchronization on concurrent algorithms , 2015, PPoPP.

[21]  Sally Fincher,et al.  Computer Science Curricula 2013 , 2013 .

[22]  Joint Task Force on Computing Curricula.  Computer Science Curricula 2013: Curriculum Guidelines for Undergraduate Degree Programs in Computer Science , 2013 .

[23]  Irena Koprinska,et al.  Students at Risk: Detection and Remediation , 2015, EDM.