Key Findings from recent literature on Computer-aided Assessment

A perusal of the literature on Computer-aided Assessment (CAA) yields a rich crop of case studies and ‘should-do’s. This paper gathers together the key findings and common themes found in a search of recent papers published on CAA implementation, including some projects under the Teaching and Learning Technology Programme (TLTP) and the Fund for the Development of Teaching and Learning (FDTL) projects. It is hoped that this review will provide a valuable snapshot of current practice in CAA across different subject disciplines and a distillation of some of the key commonalities. Initial findings indicate that barriers to embedding CAA in institutions include a lack of commitment at a strategic level, and as well as barriers in terms of cultural or pedagogical issues, rather more than any technical shortcomings in the CAA system. Furthermore, computer-assisted assessment in general is often still something of an add-on, and is not yet fully integrated into other learning and teaching practices. Technical and cultural barriers to the adoption of CAA were found to vary between subject boundaries (CAA Centre, 2001). The enablers to successful embedding of CAA in institutions include institutional commitment together with provision of adequate funds, time and personnel (Bull, 2002), frequent communication between the various stakeholders (Raine, 1999), interoperability and the existing availability of a Virtual Learning Environment (VLE) (Stevenson et al., 2001).

[1]  Joanna Bull,et al.  Computer-assisted assessment in higher education , 1999 .

[2]  Niall Sclater,et al.  Interoperability with CAA: does it work in practice? , 2002 .

[3]  Mariana Lilley,et al.  An evaluation of a computer adaptive test in a UK University context , 2003 .

[4]  Diana Laurillard,et al.  Rethinking University Teaching: A Framework for the Effective Use of Educational Technology , 1993 .

[5]  Matthew R. Wawrzynski.  Creating Significant Learning Experiences (review) , 2004 .

[6]  Joanna Bull,et al.  Assessing student learning in higher education , 1997 .

[7]  P. Davies.  “There’s no Confidence in Multiple-Choice Testing, ……” , 2002 .

[8]  Ros L O'Leary,et al.  Wading through treacle: CAA at the University of Bristol , 2001 .

[9]  Michael McCabe,et al.  CAA scoring strategies for partial credit and confidence levels , 2003 .

[10]  Joanna Bull,et al.  Quality assurance of computer‐assisted assessment: practical and strategic issues , 2000 .

[11]  Douglas M. Walker,et al.  A Note on Multiple Choice Exams, with Respect to Students' Risk Preference and Confidence , 2001 .

[12]  Niall Sclater,et al.  User requirements of the "ultimate" online assessment engine , 2003, Comput. Educ..

[13]  Banesh Hoffmann.  Multiple-choice tests , 1967 .

[14]  Joanna Bull,et al.  Computer Assisted Assessment Centre Update , 2001 .

[15]  Hugh C. Davis,et al.  Rethinking assessment through learning technologies , 2002, ASCILITE.

[16]  Emma Duke-Williams,et al.  Using computer-aided assessment to test higher level learning outcomes , 2001 .

[17]  Benjamin S. Bloom,et al.  Taxonomy of Educational Objectives: The Classification of Educational Goals. , 1957 .

[18]  J. S. Paterson.  What's in a name? - a new hierarchy for question types , 2002 .

[19]  Benjamin S. Bloom,et al.  A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives , 2000 .

[20]  Michael Burrows,et al.  Formative assessment in engineering using "TRIADS" software , 2002 .

[21]  David O'Hare,et al.  Item selection and application in Higher Education , 2002 .

[22]  Joanna Bull.  Update on the National TLTP3 Project: The implementation and evaluation of computer-assisted assessment , 1999 .

[23]  Ronald Barnett,et al.  Higher Education: A Critical Business , 1997 .

[24]  Peter G. Taylor.  Institutional change in uncertain times: Lone ranging is not enough , 1998 .

[25]  Gráinne Conole,et al.  CAA in UK HEIs – the state of the art? , 2003 .

[26]  J. Raine.  Towards the introduction of institution wide computer assisted assessment: a service department experience , 1999 .

[27]  Myles Danson,et al.  Large Scale Implementation of Question Mark Perception (V2.5). Experiences at Loughborough University , 2001 .

[28]  M. Oliver,et al.  A pedagogical framework for embedding C&IT into the curriculum , 1998 .

[29]  Chris Ricketts,et al.  Is computer-based assessment good for students? , 2001 .

[30]  C. Hart.  Doing a literature review: releasing the social science research imagination. , 1998 .

[31]  Jon H Sims Williams,et al.  Computer-Assisted Assessment in Higher Education , 1999 .

[32]  L. van Rentergem,et al.  A campus wide setup of Question Mark Perception (V2.5) at the Katholieke Universiteit Leuven (Belgium) – facing a large scale implementation , 2002 .

[33]  A. R. Gardner-Medwin,et al.  Formative and summative confidence- based assessment , 2003 .

[34]  Diana Laurillard,et al.  Rethinking University Teaching: A Conversational Framework for the Effective Use of Learning Technologies. 2nd Edition , 1993 .

[35]  H. Best.  Evaluating the introduction of CBA into the learning, teaching and assessment strategy of the diagnostic radiography course at Sheffield Hallam University , 2002 .

[36]  A. Stevenson,et al.  Integrating CAA within the University of Ulster , 2002 .

[37]  Stan Zakrzewski,et al.  A Model for Computer-based Assessment: The catherine wheel principle , 2000 .

[38]  Joanna Bull,et al.  Pebbles in the Pond: Evaluation of the CAA Centre , 2002 .

[39]  D. McPhee,et al.  Multiple choice for honours-level students? A statistical evaluation , 1999 .

[40]  Ian Hesketh,et al.  Multiple response questions – allowing for chance in authentic assessments , 2003 .

[41]  Niall Sclater,et al.  Question and test interoperability: an update on national and international developments , 2000 .

[42]  Paul Gathercoal.  Principles of Assessment , 1995 .

[43]  Su White,et al.  Creating large-scale test banks: a briefing for participative discussion of issues and agendas , 2000 .