Methodology for Evaluating a Simulator's Capability to Support Assessment of Mariner Proficiency

The goal of this project was to develop a structured process for evaluating the capability of simulators to support mariner performance assessment. In this approach, simulator evaluation objectives and criteria are based on performance-based mariner assessment requirements. The approach produces a simulator evaluation protocol that addresses four areas of simulator capability: (1) providing flexible exercise programming to the assessor; (2) replicating the characteristics of actual equipment; (3) simulating the operational conditions required to demonstrate the desired mariner performance; and (4) supporting thorough debriefing of the assessment candidate. A case study showed that our evaluation process was feasible. The process proved to be both sensitive and valuable in differentiating the capabilities of two simulators used to support mariner assessment objectives. This approach can be generalized to the evaluation of simulators of equipment used in other industries.