Towards progress indicators for measuring student programming effort during solution development

When learning to program, assignment feedback can easily reinforce a fixed mindset---where one believes intelligence is a fixed ability you either have or you don't. However, this can have negative consequences for learning. The alternative is a growth mindset, where one believes intelligence is malleable and can be improved through practice, effort, and hard work. We develop a set of fifteen progress indicators that can be used to assess student programming effort independently of the correctness of their code. The goal is to provide the measurement support needed for innovative feedback that gives a more welcoming experience for students, recognizing the effort they put in and the accomplishments they make as they work on solutions, rather than simply looking at whether code "works". Our set of progress indicators includes seven associated with writing solution code for a problem that are suitable for use on all programming assignments, and eight associated with self-checking programs using software tests, which are appropriate for assignments where students are required to test their own work. In this initial work, we applied these indicators to a collection of programming assignments from 257 students to determine the suitability of the measures and validate their results. The resulting set of progress indicators is the first step toward developing feedback strategies that recognize and reward effort and hard work, with the goal of fostering the development of a growth mindset among students.