On expert judgements in safety analyses in the process industries

Abstract Starting from logical tree structures, used in probabilistic risk analysis (PRA) and human reliability analysis (HRA), this study first presents some basic perspectives on probability and probability judgements. It is pointed out that human judgements are used in the choice of logical tree structures, classification of events and in parameter estimates. The psychological literature on expert judgment is briefly reviewed and related to engineering judgement in PRAs and HRAs. Different types of judgements and biases affecting them are identified, as well as ways of eliminating or reducing the effects of these biases. The methods of THERP, HCR and SLIM are examined from an expert judgements perspective. The results indicate that existing methods for eliciting probability judgments through interpolation between correctly determined reference points may produced quite reasonable results, if potential judgmental biases are controlled in an adequate way. In the case of judgmental extrapolation, several biases are likely to affect expert judgments thereby detracting from their validity. The review indicates that, although we have progressed towards the goal of unbiased analyses, continued research and developments are necessary for improving existing methods for safety analyses of the process industries.

[1]  E. Poulton.  Models for biases in judging sensory magnitude. , 1979, Psychological bulletin.

[2]  A. J. Spurgin,et al.  Systematic Human Action Reliability Procedure (SHARP) , 1984 .

[3]  Crystal L. Park,et al.  Intrinsic religiousness and religious coping as life stress moderators for Catholics versus Protestants. , 1990, Journal of personality and social psychology.

[4]  B. Fischhoff,et al.  Reasons for confidence. , 1980 .

[5]  Paul E. Meehl,et al.  Clinical Versus Statistical Prediction: A Theoretical Analysis and a Review of the Evidence , 1996 .

[6]  Baruch Fischhoff,et al.  When Do Base Rates Affect Predictions , 1981 .

[7]  Ali Mosleh,et al.  A critique of current practice for the use of expert opinions in probabilistic risk assessment , 1988 .

[8]  P. Hoffman.  The paramorphic representation of clinical judgment. , 1960, Psychological bulletin.

[9]  David Magnusson,et al.  THE GENERALITY OF BEHAVIORAL DATA III: GENERALIZATION POTENTIAL AS A FUNCTION OF THE NUMBER OF OBSERVATION INSTANCES , 1969 .

[10]  Ali Mosleh.  Hidden sources of uncertainty: Judgment in the collection and analysis of data , 1986 .

[11]  A. H. Murphy,et al.  “Good” Probability Assessors , 1968 .

[12]  Stuart Oskamp,et al.  The relationship of clinical experience and training methods to several criteria of clinical prediction. , 1962 .

[13]  James Shanteau,et al.  Psychological characteristics of expert decision makers , 1987 .

[14]  P. Slovic.  Analyzing the expert judge: A descriptive study of a stockbroker's decision process. , 1969 .

[15]  W. Edwards,et al.  Decision Analysis and Behavioral Research , 1986 .

[16]  JOHN COHEN,et al.  Evaluation of Compound Probabilities in Sequential Choice , 1971, Nature.

[17]  H. Helson,et al.  Adaptation-level theory , 1964 .

[18]  S. Oskamp.  OVERCONFIDENCE IN CASE-STUDY JUDGMENTS. , 1965, Journal of consulting psychology.

[19]  J. S. Wiggins,et al.  Personality and Prediction: Principles of Personality Assessment , 1973 .

[20]  John B. Kidd,et al.  The utilization of subjective probabilities in production planning , 1970 .

[21]  R. Dawes.  A case study of graduate admissions: Application of three principles of human decision making. , 1971 .

[22]  Ali Mosleh,et al.  A critique of recent models for human error rate assessment , 1988 .

[23]  H. P. Hänni,et al.  Subjective bias in PRA — The role of judgement in the selection of plant modeling input data for establishing safety goals , 1986 .

[24]  Alan D. Swain.  FIELD CALIBRATED SIMULATION. , 1967 .

[25]  Elizabeth C. Hirschman,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[26]  Lola L. Lopes,et al.  Test of an ordering hypothesis in risky decision making , 1980 .

[27]  B. Fischhoff,et al.  Diagnosticity and the base-rate effect , 1984, Memory & cognition.

[28]  M. K. Comer,et al.  Human Reliability Data Bank: evaluation results , 1985 .

[29]  E C Poulton,et al.  Biases in quantitative judgements. , 1982, Applied ergonomics.

[30]  H. J. Einhorn.  Expert judgment: Some necessary conditions and an example. , 1974 .

[31]  A. Tversky,et al.  The framing of decisions and the psychology of choice. , 1981, Science.

[32]  H. G. MADAN.  On Cases of the Production of “Ohm's (or Langberg's) Ellipses” by Biaxial Crystals , 1885, Nature.

[33]  L. J. Chapman,et al.  Illusory correlation as an obstacle to the use of valid psychodiagnostic signs. , 1969, Journal of abnormal psychology.

[34]  S. Fiske,et al.  The Handbook of Social Psychology , 1935 .

[35]  J. Carroll.  The Effect of Imagining an Event on Expectations for the Event: An Interpretation in Terms of the Availability Heuristic. , 1978 .

[36]  Annick Carnino.  Role of data and judgment in modeling human errors , 1986 .

[37]  R. Abelson.  Decision making and decision theory , 1985 .

[38]  R. Dawes,et al.  Linear models in decision making. , 1974 .

[39]  A. D. Swain,et al.  Handbook of human-reliability analysis with emphasis on nuclear power plant applications. Final report , 1983 .

[40]  A. Tversky,et al.  On the psychology of prediction , 1973 .

[41]  George Apostolakis.  On the assessment of human error rates using operational experience , 1985 .

[42]  P. Slovic,et al.  An analysis-of-variance model for the assessment of configural cue utilization in clinical judgment. , 1968, Psychological bulletin.

[43]  A. Poucet.  Survey of methods used to assess human reliability in the human factors reliability Benchmark exercise , 1988 .

[44]  B. Fischhoff,et al.  Fault trees: Sensitivity of estimated failure probabilities to problem representation. , 1978 .

[45]  Peter Kafka,et al.  Panel discussions at the international seminar on the role of data and judgment in probabilistic risk and safety analysis , 1986 .

[46]  H. S. Blackman,et al.  Evaluation of human error estimation for nuclear power plants , 1987 .

[47]  Ola Svenson,et al.  Time Perception And Long-Term Risks , 1984 .

[48]  L. R. Goldberg.  Simple models or simple processes? Some research on clinical judgments. , 1968, The American psychologist.

[49]  Norman R. Draper,et al.  Applied Regression Analysis , 1968 .

[50]  Allen Parducci,et al.  Category rating scales: Effects of relative spacing and frequency , 1971 .