Improving Teaching and Learning through Automated Short-Answer Marking

Automated short-answer marking cannot “guarantee” 100 percent agreement between the marks generated by a software system and marks produced separately by a human. This problem has prevented automated marking systems from being used in high-stake short-answer marking. This paper describes how an automated short-answer marking system, called IndusMarker, can be effectively used to improve teaching and learning in an Object-Oriented Programming course taught at a university. The system is designed for factual answers where there is a clear criterion for answers being right or wrong. The system is based on structure matching, i.e., matching a prespecified structure, developed via a purpose-built structure editor, with the content of the student's answer text.

[1]  Tom Mitchell,et al.  Towards robust computerised marking of free-text responses , 2002 .

[2]  Andrew Watt,et al.  Professional Xsl , 2001 .

[3]  M. McDaniel,et al.  Testing the testing effect in the classroom , 2007 .

[4]  Stephen G. Pulman,et al.  Automatic Short Answer Marking , 2005, ACL 2005.

[5]  Peter J. B. King,et al.  Experiences With Online Programming Examinations , 2008, IEEE Transactions on Education.

[6]  J. T. Dillon.  Questioning and Teaching: A Manual of Practice , 1988 .

[7]  Steven Holzner.  Inside XML , 2000 .

[8]  Charlie Daly,et al.  An automated learning system for Java programming , 2004, IEEE Transactions on Education.

[9]  Christopher D. Manning,et al.  Generating Typed Dependency Parses from Phrase Structure Parses , 2006, LREC.

[10]  Low Sew Ming.  An ICT-Based Technique for Assessment of Short-Answer Question , 2006, 2006 7th International Conference on Information Technology Based Higher Education and Training.

[11]  James H. McMillan,et al.  Classroom Assessment: Principles and Practice for Effective Instruction , 1996 .

[12]  F. Marton,et al.  ON QUALITATIVE DIFFERENCES IN LEARNING: I—OUTCOME AND PROCESS* , 1976 .

[13]  David Gulbransen.  The Complete Idiot's Guide to XML , 2000 .

[14]  Brett D. McLaughlin,et al.  Java and XML , 2000, The Java series.

[15]  Ethan Cerami,et al.  Building Xml Applications , 1999 .

[16]  R. Siddiqi,et al.  A systematic approach to the automated marking of short-answer questions , 2008, 2008 IEEE International Multitopic Conference.

[17]  Martin Chodorow,et al.  C-rater: Automated Scoring of Short-Answer Questions , 2003, Comput. Humanit..

[18]  George G. Karady,et al.  Strategies, Challenges and Prospects for Active Learning in the Computer-Based Classroom , 2009, IEEE Transactions on Education.

[19]  Wilfred J. Hansen,et al.  Creation of hierarchic text with a computer display , 1971 .