Computer‐based assessment in safety‐critical industries: the case of shipping

Vocational education and training (VET) concerns the cultivation and development of specific skills and competencies, in addition to broad underpinning knowledge relating to paid employment. VET assessment is, therefore, designed to determine the extent to which a trainee has effectively acquired the knowledge, skills, and competencies required by employers. Across a range of occupations, such testing has traditionally comprised a portfolio of practical on‐the‐job assessments, oral tests and laboratory tasks. In recent years vocational training assessment has increasingly adopted new computer‐based assessment (CBA) methods and, as a consequence, consideration of the scope of computer‐based assessments and their appropriateness for vocational training evaluation is timely. This paper examines the growing adoption of CBA within the safety‐critical field of maritime education and training (MET), particularly, in relation to the summative assessment of seafarers for licensing purposes. It discusses the implications for validity, reliability and security.

[1]  Michael Bloor,et al.  Keywords in Qualitative Methods , 2006 .

[2]  David O'Hare,et al.  Finding appropriate methods to assure quality Computer-Based Assessment development in UK Higher Education , 2003 .

[3]  Tony Lane,et al.  The Political Economy of Fraud in a Globalised Industry: The Case of Seafarers' Certifications , 2004 .

[4]  M. Fitch,et al.  Using high-fidelity emergency simulation with large groups of preclinical medical students in a basic science course , 2007, Medical teacher.

[5]  Stan Zakrzewski,et al.  A Model for Computer-based Assessment: The catherine wheel principle , 2000 .

[6]  Ayman Kayssi,et al.  Web‐based tutoring and testing in a computer networks course , 1999 .

[7]  A. N. Cockcroft,et al.  International convention on standards of training, certification and watchkeeping for seafarers, 1978, as amended , 2012 .

[8]  Paul A. Kirschner,et al.  Evaluating assessment quality in competence-based education: A qualitative comparison of two frameworks , 2007 .

[9]  S. Haynes,et al.  Content validity in psychological assessment: A functional approach to concepts and methods. , 1995 .

[10]  Stefan Kluj.  Usability criteria for simulators applied in the maritime engineering education , 2003 .

[11]  Edward G. Carmines,et al.  Reliability and Validity Assessment , 1979 .

[12]  B. Bloom,et al.  Taxonomy of Educational Objectives. Handbook I: Cognitive Domain , 1966 .

[13]  Jan Herrington,et al.  Authentic Assessment and Multimedia: how university students respond to a model of authentic assessment , 1998 .

[14]  M. Thoennessen,et al.  Computer-assisted assignments in a large physics class , 1996, Comput. Educ..

[15]  D. Nicol.  E‐assessment by design: using multiple‐choice tests to good effect , 2007 .

[16]  Stephen G. Sireci,et al.  On Validity Theory and Test Validation , 2007 .

[17]  Victor Gekara,et al.  Understanding attrition in UK maritime education and training , 2009 .

[18]  Derek Stephens,et al.  Computer‐assisted Assessment: suggested guidelines for an institutional strategy , 1998 .

[19]  Brian Jolly,et al.  Assuring the quality of high-stakes undergraduate assessments of clinical competence , 2006, Medical teacher.

[20]  Gráinne Conole,et al.  A review of computer-assisted assessment , 2005 .

[21]  Danny Saunders,et al.  Computer-Assisted Assessment: Staff Viewpoints on its Introduction Within a New University , 2002 .

[22]  D. Lloyd,et al.  The Introduction of Computer-Based Testing on an Engineering Technology Course. , 1996 .

[23]  S. Messick.  Meaning and Values in Test Validation: The Science and Ethics of Assessment , 1989 .

[24]  Maddalena Taras,et al.  Assessment for learning: understanding theory to improve practice , 2007 .

[25]  Mike Thelwall.  A unique style of computer‐assisted assessment , 1998 .

[26]  Robert W. Lissitz,et al.  A Suggested Change in Terminology and Emphasis Regarding Validity and Education , 2007 .

[27]  Maarten van Wesel,et al.  The influence of portfolio media on student perceptions and learning outcomes , 2008 .

[28]  Anne Wade,et al.  An Electronic Portfolio to Support Learning , 2005 .

[29]  C. Steven,et al.  Computer-based assessment: quality assurance issues, the hub of the wheel , 2003 .

[30]  Martin Bush,et al.  A Multiple Choice Test that Rewards Partial Knowledge , 2001 .

[31]  E. Lim,et al.  Computer-based testing of the modified essay question: the Singapore experience , 2007, Medical teacher.

[32]  Paul A. Kirschner,et al.  Authentic assessment, student and teacher perceptions: the practical value of the five‐dimensional framework , 2006 .

[33]  Mike Thelwall,et al.  Computer-based assessment: a versatile educational tool , 2000, Comput. Educ..

[34]  Joanna Bull,et al.  The Mass Implementation and Evaluation of Computer‐based Assessments , 1998 .