Defining the methodological challenges and opportunities for an effective science of sociotechnical systems and safety

An important part of the application of sociotechnical systems theory (STS) is the development of methods, tools and techniques to assess human factors and ergonomics workplace requirements. We focus in this paper on describing and evaluating current STS methods for workplace safety, as well as outlining a set of six case studies covering the application of these methods to a range of safety contexts. We also describe an evaluation of the methods in terms of ratings of their ability to address a set of theoretical and practical questions (e.g. the degree to which methods capture static/dynamic aspects of tasks and interactions between system levels). The outcomes from the evaluation highlight a set of gaps relating to the coverage and applicability of current methods for STS and safety (e.g. coverage of external influences on system functioning; method usability). The final sections of the paper describe a set of future challenges, as well as some practical suggestions for tackling these. Practitioner Summary: We provide an up-to-date review of STS methods, a set of case studies illustrating their use and an evaluation of their strengths and weaknesses. The paper concludes with a ‘roadmap’ for future work.

[1]  Nancy J. Cooke,et al.  Designing a Synthetic Task Environment , 2017 .

[2]  E. Hollnagel,et al.  The Importance of Functional Interdependencies in Financial Services Systems , 2017 .

[3]  Emilie M. Roth,et al.  AIDING HUMAN PERFORMANCE I : COGNITIVE ANALYSIS , 2016 .

[4]  Felix Hueber,et al.  Handbook Of Human Factors And Ergonomics Methods , 2016 .

[5]  Petra Kaufmann,et al.  Experimental And Quasi Experimental Designs For Research , 2016 .

[6]  Monika Richter,et al.  Cognition In The Wild , 2016 .

[7]  P. Nick.  From information to knowledge , 2016, Protoplasma.

[8]  Neville A Stanton,et al.  Designing sociotechnical systems with cognitive work analysis: putting theory back into practice , 2015, Ergonomics.

[9]  Neville A Stanton,et al.  Representing distributed cognition in complex systems: how a submarine returns to periscope depth , 2014, Ergonomics.

[10]  Ken Eason,et al.  Afterword: the past, present and future of sociotechnical systems theory. , 2014, Applied ergonomics.

[11]  Patrick Waterson,et al.  Crossing levels in systems ergonomics: a framework to support 'mesoergonomic' inquiry. , 2014, Applied ergonomics.

[12]  John R Wilson,et al.  Fundamentals of systems ergonomics/human factors. , 2014, Applied ergonomics.

[13]  G. McCarthy.  FRAM: the functional resonance analysis method, modelling complex socio-technical systems , 2013 .

[14]  Eric Schlosser.  Command and Control , 2013 .

[15]  Ann M. Bisantz,et al.  Cognitive Work Analysis , 2013 .

[16]  Michael C. Hout,et al.  Multidimensional Scaling , 2003, Encyclopedic Dictionary of Archaeology.

[17]  Guy H. Walker,et al.  Human Factors Methods: A Practical Guide for Engineering and Design , 2012 .

[18]  Nancy J. Cooke,et al.  Measuring Patterns in Team Interaction Sequences Using a Discrete Recurrence Approach , 2012, Hum. Factors.

[19]  Nancy J. Cooke,et al.  Preface to the Special Section on Methods for the Analysis of Communication , 2012, Hum. Factors.

[20]  E. Hollnagel.  FRAM: The Functional Resonance Analysis Method: Modelling Complex Socio-technical Systems , 2012 .

[21]  B. Karsh,et al.  Self-reported violations during medication administration in two paediatric hospitals , 2012, BMJ quality & safety.

[22]  M. Weisbord.  Productive Workplaces: Dignity, Meaning, and Community in the 21st Century , 2012 .

[23]  Robert L. Wears,et al.  Underground adaptations: case studies from health care , 2011, Cognition, Technology & Work.

[24]  Mica R. Endsley,et al.  Designing for Situation Awareness : An Approach to User-Centered Design , 2003 .

[25]  Jason J. Saleem,et al.  Redesign of a computerized clinical reminder for colorectal cancer screening: a human-computer interaction evaluation , 2011, BMC Medical Informatics Decis. Mak..

[26]  Ben-Tzion Karsh,et al.  Macroergonomics and patient safety: the impact of levels on theory, measurement, analysis and intervention in patient safety research. , 2010, Applied ergonomics.

[27]  A. Williamson,et al.  Systems-based accident analysis in the led outdoor activity domain: application and evaluation of a risk management framework , 2010, Ergonomics.

[28]  Ronald L. Boring,et al.  Lessons Learned on Benchmarking from the International Human Reliability Analysis Empirical Study , 2010 .

[29]  Neville A Stanton,et al.  From ethnography to the EAST method: A tractable approach for representing distributed cognition in Air Traffic Control , 2010, Ergonomics.

[30]  Guy H. Walker,et al.  A systemic approach to accident analysis: A case study of the Stockwell shooting , 2010, Ergonomics.

[31]  E. Hollnagel,et al.  The Changing Nature Of Risks , 2010 .

[32]  Jason J. Saleem,et al.  Provider Perceptions of Colorectal Cancer Screening Clinical Decision Support at Three Benchmark Institutions , 2009, AMIA.

[33]  Patrick Waterson,et al.  A systems ergonomics analysis of the Maidstone and Tunbridge Wells infection outbreaks , 2009, Ergonomics.

[34]  Patrick Waterson,et al.  A critical review of the systems approach within patient safety research , 2009, Ergonomics.

[35]  M. Robertson,et al.  Workplace Health Protection and Promotion through Participatory Ergonomics: An Integrated Approach , 2009, Public health reports.

[36]  Julia Pitsopoulos,et al.  Understanding safety and production risks in rail engineering planning and protection , 2009, Ergonomics.

[37]  Hal W. Hendrick,et al.  Macroergonomics: Theory, Methods, and Applications , 2009 .

[38]  Nancy J. Cooke,et al.  Interaction-Based Measures of Cognitive Systems , 2009 .

[39]  Karel Brookhuis,et al.  Handbook of Human Factors and Ergonomics Methods , 2009 .

[40]  Catherine M. Burns,et al.  A Cognitive Work Analysis of Cardiac Care Nurses Performing Teletriage , 2009 .

[41]  B. Kleiner,et al.  Using variance analysis to detect mismatches in role expectations in patient physician interactions in obstetric work systems , 2009 .

[42]  Emilie Roth,et al.  Technology implications of a cognitive task analysis for locomotive engineers , 2009 .

[43]  Patrick F. Reidy.  An Introduction to Latent Semantic Analysis , 2009 .

[44]  Emilie M. Roth,et al.  Capturing the Context of Use to Inform System Design , 2008 .

[45]  William C. Elm,et al.  Integrating Cognitive Systems Engineering Throughout the Systems Engineering Process , 2008 .

[46]  Catherine M. Burns,et al.  Evaluation of Ecological Interface Design for Nuclear Process Control: Situation Awareness Effects , 2008, Hum. Factors.

[47]  Neville A. Stanton,et al.  Modelling Command and Control: Event Analysis of Systemic Teamwork , 2008 .

[48]  Yueng-Hsiang Huang,et al.  Flexible workspace design and ergonomics training: impacts on the psychosocial work environment, musculoskeletal health, and work effectiveness among knowledge workers. , 2008, Applied ergonomics.

[49]  Sandra Knapp,et al.  Command and control , 2008, Nature.

[50]  Brian M. Kleiner,et al.  Macroergonomics: Work System Analysis and Design , 2008, Hum. Factors.

[51]  Neville A. Stanton,et al.  Task Analysis for Error Identification , 2008 .

[52]  Emilie M. Roth,et al.  Uncovering the Requirements of Cognitive Work , 2008, Hum. Factors.

[53]  Neville A. Stanton,et al.  Distributed situation awareness in an Airborne Warning and Control System: application of novel ergonomics methodology , 2007, Cognition, Technology & Work.

[54]  Shelly Scott-Nash,et al.  Simulation of Service Systems , 2007 .

[55]  Ann M. Bisantz,et al.  Analysis of Cognitive Work , 2007 .

[56]  G. A. Jamieson,et al.  Ecological Interface Design for Petrochemical Process Control: An Empirical Assessment , 2007, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[57]  Peter W. Foltz,et al.  Predicting Situation Awareness from Team Communications , 2007 .

[58]  W. Elm,et al.  Applied Cognitive Work Analysis: A Pragmatic Methodology for Designing Revolutionary Cognitive Affordances , 2003 .

[59]  Richard W. Pew,et al.  Human-system integration in the system development process : a new look , 2007 .

[60]  George R. S. Weir,et al.  ON MAN-MACHINE SYSTEMS , 2007 .

[61]  Robert L. Wears,et al.  Information flow during crisis management: challenges to coordination in the emergency operations center , 2007, Cognition, Technology & Work.

[62]  David B. Kaber,et al.  Using multiple cognitive task analysis methods for supervisory control interface design in high-throughput biological screening processes , 2006, Cognition, Technology & Work.

[63]  Guy H. Walker,et al.  Event analysis of systemic teamwork (EAST): a novel integration of ergonomics methods to analyse C4i activity , 2006, Ergonomics.

[64]  N A Stanton,et al.  Distributed situation awareness in dynamic systems: theoretical development and application of an ergonomics methodology , 2006, Ergonomics.

[65]  Neville A Stanton,et al.  Command and control in emergency services operations: a social network analysis , 2006, Ergonomics.

[66]  Pascale Carayon,et al.  Human factors of complex sociotechnical systems. , 2006, Applied ergonomics.

[67]  Emilie M. Roth,et al.  Shared Situation Awareness as a Contributor to High Reliability Performance in Railroad Operations , 2006 .

[68]  Nancy J. Cooke,et al.  2. UAV Human Factors: Operator Perspectives , 2006 .

[69]  Neelam Naikar.  Beyond interface design: Further applications of cognitive work analysis , 2006 .

[70]  Waldemar Karwowski.  Micro- and Macroergonomic Interventions in Industrially Developing Countries , 2006 .

[71]  Michelle L. Rogers,et al.  Compliance With Intended Use of Bar Code Medication Administration in Acute and Long-Term Care: An Observational Study , 2006, Hum. Factors.

[72]  David Woods,et al.  Resilience Engineering: Concepts and Precepts , 2006 .

[73]  Caprice K. Christian,et al.  A prospective study of patient safety in the operating room. , 2006, Surgery.

[74]  Gary Klein,et al.  Working Minds: A Practitioner's Guide to Cognitive Task Analysis , 2006 .

[75]  B. L. William Wong,et al.  The critical decision method , 2006 .

[76]  Brian M Kleiner,et al.  Macroergonomics: analysis and design of work systems. , 2006, Applied ergonomics.

[77]  Michelle M Robertson,et al.  Effect of a workplace design and training intervention on individual performance, group effectiveness and collaboration: the role of environmental control. , 2006, Work.

[78]  Winston Bennett,et al.  Team Task Analysis: Identifying Tasks and Jobs That Are Team Based , 2005, Hum. Factors.

[79]  D. Zohar,et al.  A multilevel model of safety climate: cross-level relationships between organization and group-level climates. , 2005, The Journal of applied psychology.

[80]  Emilie M. Roth,et al.  Work-centered support systems: a human-centered approach to intelligent system design , 2005, IEEE Intelligent Systems.

[81]  Kathryn Mearns,et al.  Designing and evaluating a human factors investigation tool (HFIT) for accident analysis , 2005 .

[82]  Brian M. Kleiner,et al.  Human Systems Integration Education and Training , 2005 .

[83]  C. Burns,et al.  Lessons From a Comparison of Work Domain Models: Representational Choices and Their Implications , 2004, Hum. Factors.

[84]  Jennifer E. Fowlkes,et al.  Targeted Acceptable Responses to Generated Events or Tasks (TARGETs) , 2004 .

[85]  Brian Kleiner.  Macroergonomic Analysis and Design (MEAD) , 2004 .

[86]  J. Brown.  Participatory Ergonomics (PE) , 2004 .

[87]  Michelle Robertson.  Systems Analysis Tool (SAT) , 2004 .

[88]  C Shawn Burke,et al.  Team Task Analysis , 2004 .

[89]  Gary Klein,et al.  Critical Decision Method , 2004 .

[90]  Theodore K. Courtney,et al.  A systems analysis approach to solving office work system health and performance problems , 2004 .

[91]  A. Majchrzak,et al.  Can absence make a team grow stronger? , 2004, Harvard business review.

[92]  Brian M. Kleiner,et al.  Macroergonomics as a large work-system transformation technology , 2004 .

[93]  Emilie M. Roth,et al.  Using an integrated process of data and modeling in HRA , 2004, Reliab. Eng. Syst. Saf..

[94]  P. Carayon,et al.  Reducing Workload and Increasing Patient Safety Through Work and Workspace Design , 2003 .

[95]  Robert G. Eggleston.  Work-Centered Design: A Cognitiive Engineering Approach to System Design , 2003 .

[96]  A. Shepherd,et al.  Guide to Task Analysis , 2003 .

[97]  Robert Hutton,et al.  Decision-Centered Design: Leveraging Cognitive Task Analysis in Design , 2003 .

[98]  William C. Elm,et al.  Case Studies: Applied Cognitive Work Analysis in the Design of Innovative Decision Support , 2003 .

[99]  Robert R. Hoffman,et al.  An Empirical Comparison of Methods for Eliciting and Modeling Expert Knowledge , 2002 .

[100]  Chris W. Clegg,et al.  A Sociotechnical Method for Designing Work Systems , 2002, Hum. Factors.

[101]  Waldemar Karwowski,et al.  A Macroergonomic Approach to Reducing Work-Related Injuries , 2002 .

[102]  Michelle Robertson.  Macroergonomics of Training Systems Development , 2002 .

[103]  Patrick Waterson,et al.  Participation in job redesign: An evaluation of the use of a sociotechnical tool and its impact , 2001 .

[104]  Dennis B. Beringer,et al.  Priority and Organization of Information Accessed by Pilots in Various Phases of Flight , 2001 .

[105]  Chris Baber,et al.  Using critical path analysis to model multimodal human-computer interaction , 2001, Int. J. Hum. Comput. Stud..

[106]  Barry Kirwan,et al.  Coping with accelerating socio-technical systems , 2001 .

[107]  Ben-Tzion Karsh,et al.  The efficacy of workplace ergonomic interventions to control musculoskeletal disorders: A critical analysis of the peer-reviewed literature , 2001 .

[108]  F. Guérin,et al.  Compreender o trabalho para transformá-lo: a prática da ergonomia , 2001 .

[109]  Neville A. Stanton,et al.  Analytical Prototyping of Personal Technologies: Using Predictions of Time and Error to Evaluate User Interfaces , 2001, INTERACT.

[110]  Hal W. Hendrick,et al.  Macroergonomics: An Introduction to Work System Design , 2000 .

[111]  P Carayon,et al.  Work organization and ergonomics. , 2000, Applied ergonomics.

[112]  B C Amick,et al.  Prevalence of upper extremity musculoskeletal disorders in college students. , 2000, The American journal of medicine.

[113]  C W Clegg,et al.  Sociotechnical principles for system design. , 2000, Applied ergonomics.

[114]  J Annett,et al.  A method for measuring team skills , 2000, Ergonomics.

[115]  J Rasmussen,et al.  Human factors in a dynamic information society: where are we heading? , 2000, Ergonomics.

[116]  K J Zink,et al.  Ergonomics in the past and the future: from a German perspective to an international one , 2000, Ergonomics.

[117]  G. Klein,et al.  Cognitive Task Analysis of Teams , 2000 .

[118]  Andrew Hopkins,et al.  Lessons from Longford: The ESSO Gas Plant Explosion , 2000 .

[119]  Catherine M. Burns,et al.  There Is More to Monitoring a Nuclear Power Plant than Meets the Eye , 2000, Hum. Factors.

[120]  Pascale Carayon,et al.  Sociotechnical issues in the implementation of imaging technology , 2000, Behav. Inf. Technol..

[121]  Marvin S. Cohen,et al.  Modeling and Diagnosing Domain Knowledge Using Latent Semantic Indexing , 1999, Interact. Learn. Environ..

[122]  Les Gasser,et al.  TOP-MODELER©: Supporting Complex Strategic and Operational Decisionmaking , 2000, Inf. Knowl. Syst. Manag..

[123]  I. Svedung,et al.  Proactive Risk Management in a Dynamic Society , 2000 .

[124]  Klaus-Dieter Schewe,et al.  Integrating Database and Dialogue Design , 2000, Knowledge and Information Systems.

[125]  D. Rempel,et al.  Effect of Four Computer Keyboards in Computer Users with Upper Extremity Musculoskeletal Disorders , 1999, American journal of industrial medicine.

[126]  Leo Gugerty,et al.  Developing a Simulated Uninhabited Aerial Vehicle (UAV) Task Based on Cognitive Task Analysis: Task Analysis Results and Preliminary Simulator Performance Data , 1999 .

[127]  K. J. Vicente,et al.  Cognitive Work Analysis: Toward Safe, Productive, and Healthy Computer-Based Work , 1999 .

[128]  Karen Holtzblatt,et al.  Contextual design , 1997, INTR.

[129]  Peter Checkland,et al.  Soft systems methodology : a 30-year retrospective . Soft systems methodology in action , 1999 .

[130]  B. Latour.  Pandora's Hope: Essays on the Reality of Science Studies , 1999 .

[131]  G. C. van der Veer,et al.  Groupware Task Analysis , 1999 .

[132]  P. Carayon,et al.  Theory and practice for the implementation of 'in-house', continuous improvement participatory ergonomic programs. , 1998, Applied ergonomics.

[133]  R. Hutton,et al.  Applied cognitive task analysis (ACTA): a practitioner's toolkit for understanding cognitive task demands. , 1998, Ergonomics.

[134]  Thomas D. Hardiman,et al.  A Comparative Analysis of Sagat and Sart for Evaluations of Situation Awareness , 1998 .

[135]  Andrew R.Hale and Jan Hovden,et al.  Management and culture: the third age of safety. A review of approaches to organizational aspects of safety, health and environment , 1998 .

[136]  Erik Hollnagel.  HRA — The First Generation , 1998 .

[137]  Erik Hollnagel,et al.  Cognitive reliability and error analysis method : CREAM , 1998 .

[138]  Peter W. Foltz,et al.  An introduction to latent semantic analysis , 1998 .

[139]  Ann Majchrzak,et al.  What To Do When You Can't Have It All: Toward a Theory of Sociotechnical Dependencies , 1997 .

[140]  B. Nardi.  The Use of Ethnographic Methods in Design and Evaluation , 1997 .

[141]  Marilyn L. Fox,et al.  Job Design , 1997 .

[142]  Jens Rasmussen,et al.  Risk management in a dynamic society: a modelling problem , 1997 .

[143]  T.M. Duffy,et al.  Scenario-Based Design: Envisioning Work and Technology in System Development [Book Review] , 1996, IEEE Transactions on Professional Communication.

[144]  K. Eason,et al.  Representing socio-technical systems options in the development of new forms of work organization , 1996 .

[145]  Nancy J. Cooke,et al.  Procedural Network Representations of Sequential Data , 1996, Hum. Comput. Interact..

[146]  Gary Herrin,et al.  A guide to practical human reliability assessment , 1996 .

[147]  Jens Rasmussen,et al.  Cognitive Systems Engineering , 2022 .

[148]  Gerardine DeSanctis,et al.  Capturing the Complexity in Advanced Technology Use: Adaptive Structuration Theory , 1994 .

[149]  Christian Korunka,et al.  Effects of new technologies with special regard for the implementation process per se , 1993 .

[150]  G. Klein,et al.  Decision Making in Action: Models and Methods , 1993 .

[151]  Barry Kirwan,et al.  A Guide To Task Analysis: The Task Analysis Working Group , 1992 .

[152]  Nancy J. Cooke,et al.  Network and Multidimensional Representations of the Declarative Knowledge of Human-Computer Interface Design Experts , 1992, Int. J. Man Mach. Stud..

[153]  B. Latour.  10 ''Where Are the Missing Masses? The Sociology of a Few Mundane Artifacts'' , 1992 .

[154]  Roger W. Schvaneveldt,et al.  Pathfinder associative networks: studies in knowledge organization , 1990 .

[155]  Robert Karasek,et al.  Healthy Work : Stress, Productivity, and the Reconstruction of Working Life , 1990 .

[156]  Pascale C. Sainfort,et al.  A balance theory of job design for stress reduction , 1989 .

[157]  Roberta Calderwood,et al.  Critical decision method for eliciting knowledge , 1989, IEEE Trans. Syst. Man Cybern..

[158]  J. Bartunek,et al.  The Multiple Realities and Experiences of Internal Organisation Development Consultation in Health Care , 1989 .

[159]  Joseph J. Hurrell,et al.  Control, stress, and job satisfaction in Canadian nurses , 1988 .

[160]  Emilie M. Roth,et al.  Cognitive Engineering: Human Problem Solving with Tools , 1988 .

[161]  Ann Majchrzak,et al.  The human infrastructure impact statement (HIIS) A tool for managing the effective implementation of advanced manufacturing technology , 1988 .

[162]  A. Majchrzak,et al.  The Human Side of Factory Automation: Managerial and Human Resource Strategies for Making Automation Succeed. , 1990 .

[163]  Christopher D. Wickens,et al.  Display Proximity in Multicue Information Integration: The Benefits of Boxes , 1988 .

[164]  E. M. Roth,et al.  Chapter 1 – Cognitive Systems Engineering , 1988 .

[165]  Albert Cherns,et al.  Principles of Sociotechnical Design Revisted , 1987 .

[166]  Erik Hollnagel,et al.  Mapping Cognitive Demands in Complex Problem-Solving Worlds , 1987, Int. J. Man Mach. Stud..

[167]  Renate J. Roske-Hofstrand,et al.  Cognitive networks as a guide to menu organization: An application in the automated cockpit , 1986 .

[168]  Gary Klein,et al.  Rapid Decision Making on the Fire Ground , 1986 .

[169]  J. Rassmusen,et al.  Information Processing and Human - Machine Interaction: An Approach to Cognitive Engineering , 1986 .

[170]  Roger W. Schvaneveldt,et al.  Measuring the Structure of Expertise , 1985, Int. J. Man Mach. Stud..

[171]  R. F. Griffiths,et al.  HAZOP and HAZAN: Notes on the identification and assessment of hazards : by T.A. Kletz, Institution of Chemical Engineers, Rugby, 1983, ISBN 0-85295-165-5, 81 pages, paperback, £8.00 incl. postage and packing. , 1984 .

[172]  Erik Hollnagel,et al.  Cognitive Systems Engineering: New Wine in New Bottles , 1983, Int. J. Man Mach. Stud..

[173]  D. J. Lawson,et al.  Failure Mode, Effect and Criticality Analysis , 1983 .

[174]  Gordan H. Robinson.  Accidents and sociotechnical systems: principles for design , 1982 .

[175]  A. D. Swain,et al.  Handbook of human-reliability analysis with emphasis on nuclear power plant applications. Final report , 1983 .

[176]  D. Kirkpatrick.  Techniques for evaluating training programs , 1979 .

[177]  S. G. Hart,et al.  Time estimation as a secondary task to measure workload: Summary of research , 1978 .

[178]  D. L. Landen.  Improving the quality of working life , 1977 .

[179]  Albert B. Cherns,et al.  The Principles of Sociotechnical Design , 1976 .

[180]  F. Glen.  The social psychology of organizations , 1976 .

[181]  Howard B. Lee,et al.  Foundations of Behavioral Research , 1973 .

[182]  Louis E. Davis,et al.  The coming crisis for production management: technology and organization , 1971 .

[183]  W. Bennis,et al.  The Social Psychology of Organizations , 1966 .

[184]  E. Trist,et al.  Some Social and Psychological Consequences of the Longwall Method of Coal-Getting , 1951 .

[185]  Emilie M. Roth.  USING OBSERVATIONAL STUDY AS A TOOL FOR DISCOVERY: UNCOVERING COGNITIVE AND COLLABORATIVE DEMANDS AND ADAPTIVE STRATEGIES , .