A Semi-Automatic Computer-Aided Assessment Approach for Marking and Providing Feedback Comments

Assessment is an essential part of the learning process. It is important for educators to provide detailed and reliable evaluations to students so that they can be better prepared for future studies and the workplace. Marking and providing formative feedback can be time-consuming and prone to errors especially when detailed analyses of students’ problem-solving steps are considered. A computer-aided marking and feedback support tool that aims at improving the efficiency and effectiveness of human marking may result in reduced marking time, improved consistency in marking, and improved feedback capabilities. This paper discusses a semi-automatic approach to marking problem-solving steps in the context of elementary school mathematics using analytical assessment rubrics. A prototype tool which implements the approach is described following recommendations based on research evidence in mathematics problem solving. The tool was evaluated in an observational study which compared marking-time efficiencies obtained using the technique with those obtained from marking done manually. The result suggests that the method has the potentials to facilitate broad feedback delivery, improve marking consistency and may save on marking time. The use of such marking and feedback support systems may contribute to the overall educational goal of more accurate and consistent assessment procedures.

[1]  Agnar Aamodt,et al.  Case-Based Reasoning: Foundational Issues, Methodological Variations, and System Approaches , 1994, AI Commun..

[2]  Peter Keegan,et al.  An Introduction to Educational Assessment, Measurement, and Evaluation: Improving the Quality of Teacher-Based Assessment , 2014 .

[3]  Linda A. Suskie.  Assessing Student Learning: A Common Sense Guide , 2004 .

[4]  S. Dinham.  How to Get Your School Moving and Improving: An Evidence-Based Approach , 2008 .

[5]  Ulrik Schroeder,et al.  Using Capture & Replay for Semi-automatic Assessment , 2011 .

[6]  Ulrich Kortenkamp,et al.  E-Learning Tools with Intelligent Assessment and Feedback for Mathematics Study , 2012 .

[7]  Anders Jonsson,et al.  The use of scoring rubrics: Reliability, validity, and educational consequences , 2007 .

[8]  James M. Moser,et al.  The Acquisition of Addition and Subtraction Concepts in Grades One through Three. , 1984 .

[9]  G. Conole,et al.  A review of computer-assisted assessment , 2005 .

[10]  Ian Jones,et al.  Touch Arithmetic: A process-based Computer-Aided Assessment approach for capture of problem solving steps in the context of elementary mathematics , 2014, Comput. Educ..

[11]  W. Szetela.  Evaluating Problem Solving in Mathematics. , 1992 .

[12]  Mary McGee Wood,et al.  A human-computer collaborative approach to the marking of free text answers , 2004 .

[13]  Chris Sangwin.  Computer Aided Assessment of Mathematics , 2013 .

[14]  Sarah Davies,et al.  Effective Assessment in a Digital Age , 2010 .

[15]  B. Rittle-Johnson,et al.  Conceptual and Procedural Knowledge of Mathematics : Does One Lead to the Other ? , 2004 .

[16]  P. Black,et al.  Assessment and Classroom Learning , 1998 .

[17]  John Hattie,et al.  Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement , 2008 .

[18]  E. Mory.  Feedback research revisited. , 2004 .

[19]  Jack Hollingsworth,et al.  Automatic graders for programming classes , 1960, Commun. ACM.

[20]  Randall I. Charles,et al.  How to evaluate progress in problem solving , 1987 .

[21]  David Tall,et al.  Ambiguity and flexibility: A proceptual view of simple arithmetic , 1983 .

[22]  Michael M. Richter,et al.  Case-Based Reasoning: A Textbook , 2013 .

[23]  Randy Elliot Bennett,et al.  Does It Matter if I Take My Mathematics Test on Computer? A Second Empirical Study of Mode Effects in NAEP. , 2008 .

[24]  Sally Jordan,et al.  E-assessment: Past, present and future , 2013 .

[25]  B Garfield Joan,et al.  Beyond Testing and Grading: Using Assessment To Improve Student Learning , 1994 .

[26]  Larry Ambrose,et al.  The power of feedback. , 2002, Healthcare executive.

[27]  Dannelle D. Stevens,et al.  Introduction to Rubrics: An Assessment Tool to Save Grading Time, Convey Effective Feedback, and Promote Student Learning , 2004 .