Assessing Reliability and Validity of an Instrument for Measuring Resilience Safety Culture in Sociotechnical Systems

Background Safety culture, acting as the oil necessary in an efficient safety management system, has its own weaknesses in the current conceptualization and utilization in practice. As a new approach, resilience safety culture (RSC) has been proposed to reduce these weaknesses and improve safety culture; however, it requires a valid and reliable instrument to be measured. This study aimed at evaluating the reliability and validity of such an instrument in measuring the RSC in sociotechnical systems. Methods The researchers designed an instrument based on resilience engineering principles and safety culture as the first instrument to measure the RSC. The RSC instrument was distributed among 354 staff members from 12 units of an anonymous petrochemical plant through hand delivery. Content validity, confirmatory, and exploratory factor analysis were used to examine the construct validity, and Cronbach alpha and test-retest were employed to examine the reliability of the instrument. Results The results of the content validity index and content validity ratio were calculated as 0.97 and 0.83, respectively. The explanatory factor analysis showed 14 factors with 68.29% total variance and 0.88 Kaiser-Meyer-Olkin index. The results were also confirmed with confirmatory factor analysis (relative Chi-square = 2453.49, Root Mean Square Error of Approximation = 0.04). The reliability of the RSC instrument, as measured by internal consistency, was found to be satisfactory (Cronbach α = 0.94). The results of test-retest reliability was r = 0.85, p < 0.001. Conclusion The results of the study suggest that the measure shows acceptable validity and reliability.

[1]  Teemu Reiman,et al.  Does the concept of safety culture help or hinder systems thinking in safety? , 2014, Accident; analysis and prevention.

[2]  Frank W. Guldenmund,et al.  The use of questionnaires in safety culture research – an evaluation , 2007 .

[3]  Parag A. Pathak,et al.  Massachusetts Institute of Technology , 1964, Nature.

[4]  Christian Koch,et al.  Integration, differentiation and ambiguity in safety cultures , 2004 .

[5]  Charles E. Heckler,et al.  Applied Multivariate Statistical Analysis , 2005, Technometrics.

[6]  H. Winklhofer,et al.  Relationship Quality: a literature review and research agenda , 2008 .

[7]  M. Shekari,et al.  Quantitative assessment of resilience safety culture using principal components analysis and numerical taxonomy: A case study in a petrochemical plant , 2016 .

[8]  Alessandra Re,et al.  From cognitive reliability to competence? An evolving approach to human factors and safety , 2010, Cognition, Technology & Work.

[9]  Marcos R. S. Borges,et al.  Resilience and brittleness in the offshore helicopter transportation system: The identification of constraints and sacrifice decisions in pilots' work , 2009, Reliab. Eng. Syst. Saf..

[10]  D. Rubio,et al.  Objectifying Content Validity: Conducting a Content Validity Study in Social Work Research , 2003 .

[11]  Paulo Victor Rodrigues de Carvalho,et al.  The use of Functional Resonance Analysis Method (FRAM) in a mid-air collision to understand some characteristics of the air traffic management system resilience , 2011, Reliab. Eng. Syst. Saf..

[12]  H. Marsh,et al.  In Search of Golden Rules: Comment on Hypothesis-Testing Approaches to Setting Cutoff Values for Fit Indexes and Dangers in Overgeneralizing Hu and Bentler's (1999) Findings , 2004 .

[13]  N. Pidgeon.  SAFETY CULTURE: KEY THEORETICAL ISSUES , 1998 .

[14]  Shameem Nyla.  NATIONAL COUNCIL ON MEASUREMENT IN EDUCATION , 2004 .

[15]  D Díaz-Cabrera,et al.  An evaluation of a new instrument to measure organisational safety culture values and practices. , 2007, Accident; analysis and prevention.

[16]  Gholam Abbas Shirali,et al.  Proactive Risk Assessment to Identify Emergent Risks using Functional Resonance Analysis Method (FRAM): A Case Study in an Oil Process Unit , 2013 .

[17]  Andrew Hopkins.  Safety, Culture and Risk The Organisational Causes of Disasters , 2008 .

[18]  Lars Skyttner,et al.  General Systems Theory: Problems, Perspectives, Practice , 2006 .

[19]  S. Clarke.  Perceptions of organizational safety: implications for the development of safety culture , 1999 .

[20]  Andrew Hopkins,et al.  Lessons from Longford: The ESSO Gas Plant Explosion , 2000 .

[21]  David Woods,et al.  Resilience Engineering: Concepts and Precepts , 2006 .

[22]  Erik Hollnagel,et al.  Resilience Engineering : New directions for measuring and maintaining safety in complex systems Third Progress Report , June 2007 , 2007 .

[23]  Rachna Shah,et al.  Use of structural equation modeling in operations management research: Looking back and forward ☆ , 2006 .

[24]  C. H. Lawshe.  A QUANTITATIVE APPROACH TO CONTENT VALIDITY , 1975 .

[25]  Joe Zhu,et al.  Data envelopment analysis vs. principal component analysis: An illustrative study of economic performance of Chinese cities , 1998, Eur. J. Oper. Res..

[26]  V. Ebrahimipour,et al.  A new method for quantitative assessment of resilience engineering by PCA and NT approach: A case study in a process industry , 2013, Reliab. Eng. Syst. Saf..

[27]  Cheryl Tatano Beck,et al.  The content validity index: are you sure you know what's being reported? Critique and recommendations. , 2006, Research in nursing & health.

[28]  Tarcisio Abreu Saurin,et al.  Evaluation and improvement of a method for assessing HSMS from the resilience engineering perspective: A case study of an electricity distributor , 2011 .

[29]  N. Pidgeon.  Safety Culture and Risk Management in Organizations , 1991 .

[30]  Stian Antonsen,et al.  Safety culture and the issue of power , 2009 .

[31]  Andy P. Field,et al.  Discovering Statistics Using SPSS , 2000 .

[32]  Paul Kline,et al.  A Handbook of Test Construction : Introduction to Psychometric Design , 1987 .

[33]  Rex B. Kline,et al.  Principles and Practice of Structural Equation Modeling , 1998 .

[34]  D. Segall.  The reliability of linearly equated tests , 1994 .

[35]  C Macrae.  Book Review: “Safety, Culture and Risk: The Organisational Causes of Disaster.” , 2006 .

[36]  Floor Koornneef,et al.  Resilience Safety Culture in Aviation Organisations , 2009 .

[37]  Chen-Shan Kao,et al.  Safety culture factors, group differences, and risk perception in five petrochemical plants , 2007 .

[38]  M. Lynn.  Determination and quantification of content validity. , 1986, Nursing research.

[39]  James T. Reason,et al.  Managing the risks of organizational accidents , 1997 .

[40]  Dennis L. Jackson.  Revisiting Sample Size and Number of Parameter Estimates: Some Support for the N:q Hypothesis , 2003 .

[41]  V. Ebrahimipour,et al.  Assessment of resilience engineering factors based on system properties in a process industry , 2015, Cognition, Technology & Work.

[42]  Robert L. Wears,et al.  Resilience Engineering: Concepts and Precepts , 2006, Quality and Safety in Health Care.

[43]  Trevor Kletz,et al.  Learning from Accidents , 2001 .

[44]  Wei Zheng,et al.  Research and application of the BFM-STAMP hazard analysis method , 2013, 2013 IEEE International Conference on Intelligent Rail Transportation Proceedings.

[45]  M. D. Cooper.  Towards a model of safety culture , 2000 .

[46]  Erik Hollnagel,et al.  Barriers And Accident Prevention , 2004 .

[47]  R. Charter,et al.  Sample size requirements for precise estimates of reliability, generalizability, and validity coefficients. , 1999, Journal of clinical and experimental neuropsychology.

[48]  K. WeiK..  The Collapse of Sensemaking in Organizations: The Mann Gulch Disaster , 2009, STUDI ORGANIZZATIVI.

[49]  Marianne Törner,et al.  Nordic Safety Climate Questionnaire (NOSACQ-50): A new tool for diagnosing occupational safety climate , 2011 .

[50]  Tor-Olav Nævestad,et al.  Mapping Research on Culture and Safety in High-Risk Organizations: Arguments for a Sociotechnical Understanding of Safety Culture , 2009 .

[51]  Peter Checkland,et al.  Systems Thinking, Systems Practice , 1981 .

[52]  F. Guldenmund.  The nature of safety culture: a review of theory and research , 2000 .

[53]  Peter Dieckmann,et al.  Safety culture and crisis resource management in airway management: general principles to enhance patient safety in critical airway situations. , 2005, Best practice & research. Clinical anaesthesiology.

[54]  L. Davis.  Instrument review: Getting the most from a panel of experts , 1992 .

[55]  Majid Motamedzade,et al.  Assessing resilience engineering based on safety culture and managerial factors , 2012 .