Step Complexity Measure for Emergency Operating Procedures - Determining Weighting Factors

Abstract In complex systems, such as nuclear power plants (NPPs) or airplane control systems, human error has been regarded as the primary cause of many events. Therefore, to ensure system safety, extensive effort has been made to identify the significant factors that can cause human error. According to related studies, written manuals or operating procedures are revealed as one of the important factors, and the understandability is pointed out as one of the major reasons for procedure-related human errors. Many qualitative checklists have been suggested to evaluate emergency operating procedures (EOPs) of NPPs so as to minimize procedure-related human errors. However, since qualitative evaluations using checklists have some drawbacks, a quantitative measure that can quantify the complexity of EOPs is indispensable. From this necessity, Park et al. suggested the step complexity (SC) measure to quantify the complexity of procedural steps included in EOPs. To verify the appropriateness of the SC measure, averaged step performance time data obtained from emergency training records of the loss-of-coolant accident (LOCA) and the excess steam demand event were compared with estimated SC scores. However, although averaged step performance time data and estimated SC scores show meaningful correlation, some important issues such as determining proper weighting factors have to be clarified to ensure the appropriateness of the SC measure. These were not properly dealt with due to a lack of backup data. In this paper, to resolve one of the important issues, emergency training records are additionally collected and analyzed in order to determine proper weighting factors. The total number of collected records is 66, and the training scenarios cover five emergency conditions including the LOCA, the steam generator tube rupture, the loss of all feedwater, the loss of off-site power, and the station blackout. From these records, average step performance time data are retrieved, and new weighting factors are determined by using a nonlinear regression analysis. The results show that the SC scores quantified by the new weighting factors show statistically meaningful correlation with averaged step performance time data. Thus, it can be concluded that the SC measure can represent the complexity of procedural steps included in EOPs.

[1]  G. Guesnier,et al.  Milestones in screen-based process control : Instrumentation and control in nuclear power plants , 1995 .

[2]  James A. Senn,et al.  System structure and software maintenance performance , 1989, Commun. ACM.

[3]  Wondea Jung,et al.  Development of the step complexity measure for emergency operating procedures using entropy concepts , 2001, Reliab. Eng. Syst. Saf..

[4]  Tom Kontogiannis Applying information technology to the presentation of emergency operating procedures: Implications for usability criteria , 1999, Behav. Inf. Technol..

[5]  Inder M. Soi,et al.  Software complexity: An aid to software maintainability , 1985 .

[6]  Gabrielle de Brito Towards a model for the study of written procedure following in dynamic environments , 2002, Reliab. Eng. Syst. Saf..

[7]  Chechen Liao,et al.  The impact of data models and task complexity on end-user performance: an experimental investigation , 2000, Int. J. Hum. Comput. Stud..

[8]  Elizabeth R. Towell,et al.  Aviation human factors: a framework for the new millennium , 1999 .

[9]  W. Stadelmann,et al.  State-oriented accident management and emergency procedures at Gundremmingen nuclear power plant / Zustandsorientierte Störfallbehandlung und Notfallschutz im Kernkraftwerk Gundremmingen , 1999 .

[10]  G A Sundström Towards models of tasks and task complexity in supervisory control applications. , 1993, Ergonomics.

[11]  R. L. Brune,et al.  Development of a checklist for evaluating maintenance, test and calibration procedures used in nuclear power plants. Formal technical report 14 February 1979-14 March 1980 , 1980 .

[12]  D. L. Carver Producing maintainable software , 1987 .

[13]  Roger W. Schvaneveldt,et al.  Measuring the Structure of Expertise , 1985, Int. J. Man Mach. Stud..

[14]  Jaewhan Kim,et al.  The step complexity measure for emergency operating procedures - comparing with simulation data , 2001, Reliab. Eng. Syst. Saf..

[15]  D. Campbell Task Complexity: A Review and Analysis , 1988 .

[16]  Asaf Degani,et al.  Cockpit Checklists: Concepts, Design, and Use , 1993 .

[17]  Hans-Joachim Uth Trends in major industrial accidents in Germany , 1999 .

[18]  Richard Lai,et al.  On measuring the complexity of an estelle specification , 1998, J. Syst. Softw..

[19]  Erik Hollnagel,et al.  Guidelines for computerized presentation of emergency operating procedures , 1996 .

[20]  John L. Campbell,et al.  The development of human factors design guidelines , 1996 .

[21]  David E. Kieras,et al.  An Approach to the Formal Analysis of User Complexity , 1999, Int. J. Man Mach. Stud..

[22]  Zhi-Gang Wei,et al.  A Quantitative Measure for Degree of Automation and Its Relation to System Performance and Mental Load , 1998, Hum. Factors.

[23]  Renato R. Gonzalez,et al.  A unified metric of software complexity: Measuring productivity, quality, and value , 1995, J. Syst. Softw..

[24]  Ed M. Dougherty Commentary: EOPs, a lingering concern , 1995 .

[25]  A.J. Spurgin,et al.  Testing an expert system: testing the emergency operating procedures tracking system , 1988, Conference Record for 1988 IEEE Fourth Conference on Human Factors and Power Plants,.

[26]  Shunsuke Kondo Lessons learned for PSA from the SGTR incident at Mihama, unit 2, in 1991 , 1994 .

[27]  R. B. Williamson,et al.  Safety Management Assessment System (SMAS): a process for identifying and evaluating human and organization factors in marine system operations with field test results , 1999 .

[28]  Mark Green,et al.  Optimising procedures in manufacturing systems , 1996 .

[29]  W. Lee,et al.  Including the whys and wherefores in procedural training: intelligent training for emergencies in nuclear power plants , 1991, Conference Proceedings 1991 IEEE International Conference on Systems, Man, and Cybernetics.

[30]  Yves Dien Safety and application of procedures, or `how do `they' have to use operating procedures in nuclear power plants?' , 1998 .

[31]  H. Dieter Rombach,et al.  A Controlled Expeniment on the Impact of Software Structure on Maintainability , 1987, IEEE Transactions on Software Engineering.

[32]  John W. Payne,et al.  Task complexity and contingent processing in decision making: An information search and protocol analysis☆ , 1976 .

[33]  D. D. Heea,et al.  Safety Management Assessment System ( SMAS ) : a process for identifying and evaluating human and organization factors in marine system operations with field test results , 1999 .

[34]  Philip Marsden,et al.  Procedures in the nuclear industry , 1996 .

[35]  Robert Sargent,et al.  Development and evaluation of the Maintenance Error Decision Aid (MEDA) process , 2000 .

[36]  JärvelinKalervo,et al.  Task complexity affects information seeking and use , 1995 .

[37]  H. Unger,et al.  The human error rate assessment and optimizing system HEROS - a new procedure for evaluating and optimizing the man-machine interface in PSA , 2001, Reliab. Eng. Syst. Saf..

[38]  Taghi M. Khoshgoftaar,et al.  An Information Theory-Based Approach to Quantifying the Contribution of a Software Metric , 1997, J. Syst. Softw..

[39]  S Salminen,et al.  Human errors in fatal and serious occupational accidents in Finland. , 1996, Ergonomics.

[40]  Jessica Keyes,et al.  IEEE Standard Dictionary of Measures to Produce Reliable Software , 2002 .

[41]  Gunnar Johannsen,et al.  INTERNAL REPRESENTATION, INTERNAL MODEL, HUMAN PERFORMANCE MODEL AND MENTAL WORKLOAD , 1989 .

[42]  Ali Mosleh,et al.  A methodology for modeling operator errors of commission in probabilistic risk assessment , 1994 .

[43]  Stephen S. Yau,et al.  An Integrated Life-Cycle Model for Software Maintenance , 1988, IEEE Trans. Software Eng..

[44]  V. I. Naumov The human factor and organization to support nuclear power plant operators , 1993 .

[45]  Kang Li,et al.  Understanding Perceived Complexity in Human Supervisory Control , 2000, Cognition, Technology & Work.

[46]  Henk G. Stassen,et al.  Internal representation, internal model, human performance model and mental workload , 1988, Autom..

[47]  Kalervo Järvelin,et al.  Task Complexity Affects Information Seeking and Use , 1995, Inf. Process. Manag..

[48]  Asaf Degani,et al.  Procedures in complex systems: the airline cockpit , 1997, IEEE Trans. Syst. Man Cybern. Part A.

[49]  R. Wood Task complexity: Definition of the construct , 1986 .

[50]  Frédéric Vanderhaegen,et al.  APRECIH: a human unreliability analysis method – application to railway system , 1999 .