Authoring diagram-based CBA with CourseMarker

The CourseMarker system has been used to assess free-response computer based assessment (CBA) exercises since 1998. The aim of the studies reported here was to evaluate the feasibility and usefulness of developing and deploying diagram-based exercises using DATsys, an authoring environment for diagram-based CBA, together with CourseMarker. Postgraduate students constructed diagram-based exercises in four domains. The process of constructing the exercises was captured as an indicator of feasibility. The exercises were then used to assess two cohorts of undergraduate students. Instruments including system submission logs and student questionnaires were used to assess usefulness. Findings indicate that there is considerable potential for the assessment of free-response domains such as diagrams. Such an approach can help students as part of an iterative process of learning by allowing repeated submission of coursework, which may be most appropriate within a formative assessment context. The exercises are popular with students and demonstrate a gradual, though decelerating, increase in marks over subsequent submissions. The techniques are reliable, but further development allowing for alternative model solutions and assessment of the aesthetic appearance of diagrams would increase validity. Our techniques and findings are novel for CBA, and have implications for the increasingly important research area of formative assessment.