A reliability and usability study of TRACEr-RAV: the technique for the retrospective analysis of cognitive errors--for rail, Australian version.

The aim of this study was to compare the usability and reliability of two human error identification tools: TRACEr-Rail (developed by the Rail Safety and Standards Board in the UK) and TRACEr-RAV (an Australian specific version of the tool). Following an attempt to modify TRACEr-Rail to more appropriately suit the Australian rail context, it was predicted that TRACEr-RAV would be rated as more usable and be applied more consistently by Australian users than TRACEr-Rail. In Experiment 1, twenty-five rail employees used either TRACEr-Rail or TRACEr-RAV1 to extract and classify errors from six Australian rail incident reports. In Experiment 2, eleven university students used both TRACEr-Rail and TRACEr-RAV2 to extract and classify errors from three incident summaries. The results revealed that although modification of TRACEr-Rail to become TRACEr-RAV1 and TRACEr-RAV2 did not result in improved inter-rater reliability, modification resulted in improved ratings of usability in Experiment 2. Most participants in Experiment 2 preferred TRACEr-RAV2 to TRACEr-Rail. The poor inter-rater reliability observed was most likely the result of inadequate training, limited practice in using the tools, and insufficient human factors knowledge.

[1]  Alastair Ross,et al.  Technical note: measurement issues in taxonomic reliability , 2004 .

[2]  W Huw Gibson.  T270 Phase 3: User Trial of the Rail-Specific HRA Technique , 2006 .

[3]  Scott A. Shappell,et al.  A Human Error Approach to Accident Investigation: The Taxonomy of Unsafe Operations , 1997 .

[4]  B Kirwan,et al.  Human error identification in human reliability assessment. Part 2: Detailed comparison of techniques. , 1992, Applied ergonomics.

[5]  David Sless.  Designing public documents , 2004 .

[6]  K. Catchpole,et al.  Identification of systems failures in successful paediatric cardiac surgery , 2006, Ergonomics.

[7]  Alastair Ross,et al.  Beyond Human Error: Taxonomies and Safety Science , 2006 .

[8]  Barry Kirwan,et al.  A Guide to Practical Human Reliability Assessment , 1994 .

[9]  David Sless,et al.  Designing a new bill for Telecom Australia , 1990 .

[10]  Stanley Hall.  Beyond Hidden Dangers : Railway Safety into the 21st Century , 2003 .

[11]  S. Stevenage,et al.  Learning to predict human error: issues of acceptability, reliability and validity. , 1998, Ergonomics.

[12]  D O'Hare,et al.  The ‘Wheel of Misfortune’: a taxonomic approach to human factors in accident investigation and analysis in aviation and other complex systems , 2000, Ergonomics.

[13]  Barry Kirwan,et al.  Development and application of a human error identification tool for air traffic control. , 2002, Applied ergonomics.

[14]  Melissa T. Baysari,et al.  Classification of errors contributing to rail incidents and accidents: A comparison of two human error identification techniques , 2009 .

[15]  Barry Kirwan,et al.  Validation of human reliability assessment techniques: Part 1 -- Validation issues , 1997 .

[16]  Kelley J. Krokos,et al.  Preface to the Special Section on Classifying and Understanding Human Error , 2007, Hum. Factors.