Wearing the Assessment 'BRACElet'

Introduction There has been much written about the difficulties novices experience when learning to write programs and the various strategies to overcome these problems (Dunican, 2002; Miliszewska & Tan, 2007). Despite much reporting upon the plethora of differing teaching approaches, what is not often recognized in the literature is that many students after a semester of instruction are still unable to complete a simple program-writing task (Clear et al., 2008). In their early study, Soloway et al. (1983) reported that more than three-fifths of computer programming students failed when asked to write a simple program to calculate the average of a set of numbers. In a broader and more ambitious study in 2001, several computing student populations in different countries were tested on a common set of program-writing problems (McCracken et al., 2001). This study, along with its follow-up of 615 students across 7 countries in 12 institutions, showed that the majority of computing students were still unable at the end of semester to write a simple computer program, despite the best efforts of their teachers (Lister et al., 2004). To begin to address this problem, a workshop titled Building Research in Australasian Computing Education (BRACE) was organised to run following the 2004 Australian Computing Education Conference. Funded by a SIGSCE award, the BRACE workshop brought together Australian and New Zealand computing academics interested in learning and applying the techniques and methodologies of computer science education action research to the problem of poor student codewriting performance. A brainstorming exercise was conducted to find possible effective approaches to improve on the current situation. Workshop attendees agreed to participate in a common research activity, and, at a follow-up meeting in 2005, participants organised to gather assessment data in their own classrooms. The goal was to use end-of-semester assessments to try to pinpoint the key steps and difficulties beginners faced in learning introductory programming. Each participant then committed to the addition of an agreed small common set of question types for use within their local examinations, thus repeating an earlier study by Lister et al. (2004) which investigated the relationship between novice programmers' ability to read code with codewriting performance. The subsequent analysis and discussion of the collected data of these commitments began an iterative series of action research, now known as the BRACElet project. The BRACElet project community meets biannually at workshops which are timed to coincide with key computing education research conferences. The timing enhances the opportunities for interaction, discussion, and exchange between BRACElet participants and the research community as a whole. Each year there is a continual refinement of the action research approach to see a common assessment framework being adopted across all participating institutions. The aims of the BRACElet project have widened from researchers trying to understand novice difficulties in program-writing to an attempt in building theory on how learners acquire programming knowledge. This is achieved through annual incremental changes to the agreed set of common assessment tasks and their subsequent analyses. Additionally, for participants the BRACElet project provides the benefits of external moderation, quality assurance, and benchmarking of examinations From Australasian beginnings, BRACElet has continued to attract new collaborators to become a large multi-national, multi-institutional research community. BRACElet has published over 20 papers, with 28 academics as authors, from 20 tertiary institutions across 7 countries. In 2008, academics from Victoria University joined the BRACElet group; this paper reports upon this collaborative experience and describes how it has informed local teaching practice. The following section expands upon the BRACElet project in detail, followed by a description of our local experience before commenting on both the local and the broader impacts of the BRACElet project. …