Disparate Measures in Examinations for STCW Certificates of Competency: The Use and Effectiveness of Multiple Choice Questions by

Mariners are familiar with the process of obtaining a certificate of competency through qualifying examination. Mariners know that their certificates are required to meet standards as laid out in the STCW 1995 Convention. What may not be appreciated is that examination methods are disparate, with considerable differences country to country. Maritime administrations have multi-dimensional examination regimes in varying combinations of written, practical demonstration, simulation and interview (oral). The effectiveness of the examination process is critical, since its purpose is to ensure a certificate of competence holder can complete on-board tasks safely and with the least environmental impact. There are no comparative studies questioning the relative merits of examination regimes. Embodying this disparity is the use of multiple-choice questions (MCQ), the subject of the authors‟ surveys and studies published at international conferences in Marseille, 2006; Rotterdam, 2007, and Szczecin, 2009. They describe a survey to determine the extent to which MCQ are used in STCW examinations; a survey soliciting opinions from maritime lecturers; a study to identify factors producing artificial test scores, and a study comparing the MCQ performance of student mariners relative to students with no maritime technology knowledge. MCQ are increasingly used in STCW examination. There are concerns about their effectiveness and reliability, where the authors‟ studies show that significant scores can be obtained by students with no marine technology knowledge. There are concerns regarding their place in an examination evidencing competence. There are concerns that use is driven by economics and administrative convenience. The paper reviews the authors‟ progressive research; the pros and cons and principle factors influencing MCQ assessment, with the objective of providing insights into the relative significance of these factors so that they may be taken into account. The paper offers conclusions for the consideration of maritime administrators and nautical college officials responsible for examination methods; for maritime lecturers tasked with constructing MCQ items, and for maritime students seeking an understanding of the process leading to professional qualification. The paper suggests a solution addressing the disparity question.