The Binary-Based Model (BBM) for Improved Human Factors Method Selection

OBJECTIVE This paper presents the Binary-Based Model (BBM), a new approach to Human Factors (HF) method selection. The BBM helps practitioners select the most appropriate HF methodology in relation to the complexity within the target system. BACKGROUND There are over 200 HF methods available to the practitioner and little guidance to help choose between them. METHOD The BBM defines a HF "problem space" comprising three complexity attributes. HF problems can be rated against these attributes and located in the "problem space." In addition, a similar HF "approach space" in which 66 predictive methods are rated according to their ability to confront those attributes is defined. These spaces are combined into a "utility space" in which problems and methods coexist. In the utility space, the match between HF problems and methods can be formally assessed. RESULTS The method space is split into octants to establish broad groupings of methods distributed throughout the space. About 77% of the methods reside in Octant 1 which corresponds to problems with low levels of complexity. This demonstrates that most HF methods are suited to problems in low-complexity systems. CONCLUSION The location of 77% of the rated methods in Octant 1 indicates that HF practitioners are underserved with methods for analysis of HF problems exhibiting high complexity. APPLICATION The BBM can be used by multidisciplinary teams to select the most appropriate HF methodology for the problem under analysis. All the materials and analysis are placed in the public domain for modification and consensus building by the wider HF community.

[1]  J Rogalski,et al.  Analysing communication in complex distributed decision-making. , 1993, Ergonomics.

[2]  Neelam Naikar Work Domain Analysis: Concepts, Guidelines, and Cases , 2013 .

[3]  Neville A Stanton,et al.  Representing distributed cognition in complex systems: how a submarine returns to periscope depth , 2014, Ergonomics.

[4]  Neville A. Stanton,et al.  On the reliability and validity of, and training in, ergonomics methods: a challenge revisited , 2016 .

[5]  Laurence Moseley,et al.  Knowledge elicitation: Principles, techniques and applications , 1990 .

[6]  S. Sharma,et al.  An exploratory study of chaos in human-Machine system dynamics , 2006, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[7]  D. Woods,et al.  Behind Human Error , 2010 .

[8]  Joel Cutcher-Gershenfeld,et al.  Engineering resilience into safety-critical systems , 2012 .

[9]  Mark S. Young,et al.  Guide to Methodology in Ergonomics: Designing for Human Use , 1999 .

[10]  M. M. Ayoub,et al.  Fuzzy modelling of stresses in manual lifting tasks. , 1984, Ergonomics.

[11]  Guy H. Walker,et al.  Measuring Dimensions of Command and Control Using Social Network Analysis. Extending the NATO SAS-050 model , 2009 .

[12]  E. N. Corlett,et al.  Evaluation of Human Work , 2005 .

[13]  Adam Hulme,et al.  Radical systems thinking and the future role of computational modelling in ergonomics: an exploration of agent-based modelling , 2019, Ergonomics.

[14]  Neville A Stanton,et al.  Fitting methods to paradigms: are ergonomics methods fit for systems thinking? , 2017, Ergonomics.

[15]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[16]  Sidney Dekker,et al.  Anticipating the effects of technological change: A new era of dynamics for human factors , 2000 .

[17]  Pierre Falzon,et al.  A strategy for human factors/ergonomics: developing the discipline and profession , 2012, Ergonomics.

[18]  Richard E. Hayes,et al.  Understanding Command and Control , 2011 .

[19]  B Kirwan,et al.  Human error identification techniques for risk assessment of high risk systems--part 2: towards a framework approach. , 1998, Applied ergonomics.

[20]  Danko Nikolić,et al.  SITUATION AWARENESS AS A PREDICTOR OF PERFORMANCE IN EN ROUTE AIR TRAFFIC CONTROLLERS , 1998 .

[21]  Robert L. Wears,et al.  Resilience Engineering: Concepts and Precepts , 2006, Quality and Safety in Health Care.

[22]  Gary Herrin,et al.  A guide to practical human reliability assessment , 1996 .

[23]  George E. Cooper,et al.  The use of pilot rating in the evaluation of aircraft handling qualities , 1969 .

[24]  E Salas,et al.  Task and training requirements analysis methodology (TTRAM): an analytic methodology for identifying potential training uses of simulator networks in teamwork-intensive task environments. , 1998, Ergonomics.

[25]  Neville A. Stanton,et al.  Observing the observer: non-intrusive verbalisations using the Concurrent Observer Narrative Technique , 2011, Cognition, Technology & Work.

[26]  WILLIAM S. MARRAS,et al.  A neuro-fuzzy model for estimating electromyographical activity of trunk muscles due to manual lifting , 2003, Ergonomics.

[27]  Mica R. Endsley,et al.  Designing for Situation Awareness in Complex System , 2001 .

[28]  P. Cilliers,et al.  Complexity and post-modernism: understanding complex systems , 1999 .

[29]  Nancy G. Leveson,et al.  A new accident model for engineering safer systems , 2004 .

[30]  Allen Newell,et al.  The psychology of human-computer interaction , 1983 .

[31]  Mark S. Young,et al.  Guide to Methodology in Ergonomics: Designing for Human Use , 1999 .

[32]  John D. Lee,et al.  Emerging challenges in cognitive ergonomics: Managing swarms of self-organizing agent-based automation , 2001 .

[33]  Linda J. Sorensen,et al.  Cognitive Work Analysis: Coping with Complexity , 2010 .

[34]  John R Wilson,et al.  Fundamentals of systems ergonomics/human factors. , 2014, Applied ergonomics.

[35]  D. Woods Coping with complexity: the psychology of human behaviour in complex systems , 1988 .

[36]  A Pretorius,et al.  Development of a mental workload index: A systems approach , 2007, Ergonomics.

[37]  Neville A Stanton,et al.  Hierarchical task analysis: developments, applications, and extensions. , 2006, Applied ergonomics.

[38]  Neville A Stanton,et al.  Combining network analysis with Cognitive Work Analysis: insights into social organisational and cooperation analysis , 2015, Ergonomics.

[39]  Mark S. Young,et al.  What price ergonomics? , 1999, Nature.

[40]  C Shawn Burke,et al.  Team Task Analysis , 2004 .

[41]  A. H. Roscoe,et al.  A Subjective Rating Scale for Assessing Pilot Workload in Flight: A decade of Practical Use , 1990 .

[42]  Thomas E. Nygren,et al.  The Subjective Workload Assessment Technique: A Scaling Procedure for Measuring Mental Workload , 1988 .

[43]  Riccardo Patriarca,et al.  Modelling complexity in everyday operations: functional resonance in maritime mooring at quay , 2017, Cognition, Technology & Work.

[44]  Dianne E. Howie,et al.  Measures of operator performance in complex, dynamic microworlds: advancing the state of the art. , 1998, Ergonomics.

[45]  Peter A Hancock,et al.  State of science: mental workload in ergonomics , 2015, Ergonomics.

[46]  Raja Parasuraman,et al.  Fuzzy Signal Detection Theory: Basic Postulates and Formulas for Analyzing Human and Machine Performance , 2000, Hum. Factors.

[47]  Gavriel Salvendy,et al.  Handbook of Human Factors and Ergonomics: Salvendy/Handbook of Human Factors 4e , 2012 .

[48]  Barry Kirwan,et al.  A Guide To Task Analysis: The Task Analysis Working Group , 1992 .

[49]  Xingda Qu,et al.  Human factors assessment of conflict resolution aid reliability and time pressure in future air traffic control , 2015, Ergonomics.

[50]  Paul M. Salmon,et al.  Chapter 10 The Challenges of Practice-Oriented Research , 2016 .

[51]  Neville A Stanton,et al.  Giving ergonomics away? The application of ergonomics methods by novices. , 2003, Applied ergonomics.

[52]  Per Øivind Braarud,et al.  Subjective Task Complexity and Subjective Workload: Criterion Validity for Complex Team Tasks , 2001 .

[53]  M. Mitchell Waldrop,et al.  Complexity : the emerging science and the edge of order and chaos , 1992 .

[54]  Guy H. Walker,et al.  State-of-science: situation awareness in individuals, teams and systems , 2017, Ergonomics.

[55]  A. Monk,et al.  Reasoning about tasks, activities and technology to support collaboration. , 1998, Ergonomics.

[56]  Gavriel Salvendy,et al.  Development of a human information processing model for cognitive task analysis and design , 2006 .

[57]  Mica R. Endsley,et al.  Situation awareness global assessment technique (SAGAT) , 1988, Proceedings of the IEEE 1988 National Aerospace and Electronics Conference.

[58]  Steven Shorrock,et al.  Human Factors and Ergonomics in Practice : Improving System Performance and Human Well-Being in the Real World , 2016 .

[59]  P. Tsang,et al.  Diagnosticity and multidimensional subjective workload ratings. , 1996, Ergonomics.

[60]  Gary Klein,et al.  Critical Decision Method , 2004 .

[61]  Eduardo Salas,et al.  Team workload , 2014 .

[62]  Neville A Stanton,et al.  Beyond human error taxonomies in assessment of risk in sociotechnical systems: a new paradigm with the EAST ‘broken-links’ approach , 2017, Ergonomics.

[63]  C W Clegg,et al.  Sociotechnical principles for system design. , 2000, Applied ergonomics.

[64]  Lisanne Bainbridge,et al.  Verbal protocol analysis. , 1990 .

[65]  E. Barrett,et al.  DECISION MAKING UNDER STRESS , 2018 .

[66]  G. Lintern Cognitive Systems Engineering , 2012 .

[67]  John Hunter,et al.  Functional holography analysis: simplifying the complexity of dynamical networks. , 2006, Chaos.

[68]  R. M. Taylor,et al.  Situational Awareness Rating Technique (Sart): The Development of a Tool for Aircrew Systems Design , 2017 .

[69]  R. Hutton,et al.  Applied cognitive task analysis (ACTA): a practitioner's toolkit for understanding cognitive task demands. , 1998, Ergonomics.

[70]  K. J. Vicente,et al.  Cognitive Work Analysis: Toward Safe, Productive, and Healthy Computer-Based Work , 1999 .

[71]  K. Ikemura Development and application , 1971 .

[72]  Eileen B. Entin,et al.  Questionnaires for Distributed Assessment of Team Mutual Awareness , 2004 .

[73]  S. Dekker,et al.  The complexity of failure: Implications of complexity theory for safety investigations , 2011 .

[74]  John Scott What is social network analysis , 2010 .

[75]  Guy H. Walker,et al.  Command and Control: The Sociotechnical Perspective , 2009 .

[76]  P. Salmon,et al.  Back to the future: What do accident causation models tell us about accident prediction? , 2018 .

[77]  Erik Hollnagel,et al.  Barriers And Accident Prevention , 2004 .

[78]  G. McCarthy FRAM: the functional resonance analysis method, modelling complex socio-technical systems , 2013 .

[79]  Alfred Hubler,et al.  Predicting complex systems with a holistic approach: The “throughput” criterion , 2005 .

[80]  Terje Aven,et al.  Perspectives on risk: review and discussion of the basis for establishing a unified and holistic approach , 2005, Reliab. Eng. Syst. Saf..

[81]  Guy H. Walker,et al.  Evaluation of knowledge transfer for occupational safety and health in an organizational context: development of an evaluation framework , 2016 .

[82]  Neville A Stanton,et al.  From ethnography to the EAST method: A tractable approach for representing distributed cognition in Air Traffic Control , 2010, Ergonomics.

[83]  Chris Baber,et al.  Using critical path analysis to model multimodal human-computer interaction , 2001, Int. J. Hum. Comput. Stud..

[84]  Anders Jansson,et al.  Collegial verbalisation – a case study on a new method on information acquisition , 2007, Behav. Inf. Technol..

[85]  Neville A. Stanton,et al.  East: A method for investigating social, information and task networks , 2014 .

[86]  Michael D. Harrison,et al.  THEA: A Technique for Human Error Assessment Early in Design , 2001, INTERACT.

[87]  Felix Hueber,et al.  Handbook Of Human Factors And Ergonomics Methods , 2016 .

[88]  Michael A. Vidulich,et al.  Testing a Subjective Metric of Situation Awareness , 1991 .

[89]  G. C. van der Veer,et al.  Groupware Task Analysis , 1999 .

[90]  Alfred Hubler,et al.  Understanding complex systems: Defining an abstract concept , 2007 .

[91]  Mark Torrance,et al.  Identifying the task variables that influence perceived object assembly complexity , 2004, Ergonomics.

[92]  Neville A Stanton,et al.  Translating concepts of complexity to the field of ergonomics , 2010, Ergonomics.

[93]  R L Helmreich,et al.  Cockpit Management Attitudes , 1984, Human factors.

[94]  Erik Hollnagel,et al.  Cognitive reliability and error analysis method : CREAM , 1998 .

[95]  Lena Osterhagen,et al.  Evaluation Of Human Work , 2016 .

[96]  David Klinger,et al.  Team Decision Requirement Exercise: Making Team Decision Requirements Explicit , 2004 .

[97]  Cathleen Wharton,et al.  Cognitive Walkthroughs: A Method for Theory-Based Evaluation of User Interfaces , 1992, Int. J. Man Mach. Stud..

[98]  Peter J Goillau,et al.  MALERN CAPACITY ESTIMATE (MACE): A PROPOSED COGNITIVE MEASURE FOR COMPLEX SYSTEMS. , 1997 .

[99]  Guy H. Walker,et al.  What the Death Star can tell us about ergonomics methods , 2016 .

[100]  N. Stanton,et al.  Is utility in the mind of the beholder? A study of ergonomics methods. , 1998, Applied Ergonomics.

[101]  Guy H. Walker,et al.  Human Factors Methods: A Practical Guide for Engineering and Design , 2012 .

[102]  R. F. Griffiths,et al.  HAZOP and HAZAN: Notes on the identification and assessment of hazards : by T.A. Kletz, Institution of Chemical Engineers, Rugby, 1983, ISBN 0-85295-165-5, 81 pages, paperback, £8.00 incl. postage and packing. , 1984 .

[103]  Tom A. B. Snijders,et al.  Social Network Analysis , 2011, International Encyclopedia of Statistical Science.

[104]  Edward A. Smith Complexity, Networking, & Effects-Based Approaches to Operations , 2006 .

[105]  J Sauer,et al.  Effects of training on short- and long-term skill retention in a complex multiple-task environment , 2000, Ergonomics.

[106]  J. Gibbs Elementary Principles in Statistical Mechanics: Developed with Especial Reference to the Rational Foundation of Thermodynamics , 1902 .

[107]  Chris T. Kiranoudis,et al.  A fuzzy modeling application of CREAM methodology for human reliability analysis , 2006, Reliab. Eng. Syst. Saf..

[108]  Alfred W. Hübler,et al.  Predicting complex systems with a holistic approach: The "throughput" criterion , 2005, Complex..

[109]  Gavriel Salvendy,et al.  Handbook of Human Factors and Ergonomics , 2005 .

[110]  Herbert B. Enderton,et al.  A mathematical introduction to logic , 1972 .

[111]  Mica R. Endsley,et al.  Measurement of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[112]  Neville A Stanton,et al.  Quantum ergonomics: shifting the paradigm of the systems agenda , 2017, Ergonomics.

[113]  Neville A. Stanton,et al.  Distributed situation awareness in an Airborne Warning and Control System: application of novel ergonomics methodology , 2007, Cognition, Technology & Work.

[114]  Waldemar Karwowski Cognitive Ergonomics; Requisite Compatibility, Fuzziness and Nonlinear Dynamics , 2000 .

[115]  Alfred Hubler,et al.  Information engines: Converting information into energy , 2006 .

[116]  Neville A Stanton,et al.  Editorial New paradigms in ergonomics , 2017, Ergonomics.

[117]  D. E. Embrey,et al.  SHERPA: A systematic human error reduction and prediction approach , 2013 .

[118]  Raja Parasuraman,et al.  Fuzzy signal detection theory: analysis of human and machine performance in air traffic control, and analytic considerations , 2003, Ergonomics.

[119]  Mark A. Neerincx,et al.  Cognitive task load analysis : Allocating tasks and designing support , 2003 .

[120]  Guy H. Walker,et al.  Practitioner versus analyst methods: a nuclear decommissioning case study. , 2014, Applied ergonomics.

[121]  A. Shepherd,et al.  Guide to Task Analysis , 2003 .

[122]  Zhang Li,et al.  Fuzzy logic-based approach for identifying the risk importance of human error , 2010 .

[123]  G. Walker Fortune favours the bold , 2016 .

[124]  Bruce G. Coury,et al.  Transportation Accident Investigation: The Development of Human Factors Research and Practice: , 2010 .

[125]  Neelam Naikar Work Domain Analysis , 2013 .

[126]  Kenneth James William Craik,et al.  PSYCHOLOGICAL AND PHYSIOLOGICAL ASPECTS OF CONTROL MECHANISMS WITH SPECIAL REFERENCE TO TANK GUNNERY. PART I , 1963 .

[127]  Neville A. Stanton,et al.  A SYSTEMS APPROACH TO HUMAN ERROR IDENTIFICATION , 1996 .

[128]  B Kirwan,et al.  Human error identification techniques for risk assessment of high risk systems--Part 1: Review and evaluation of techniques. , 1998, Applied ergonomics.

[129]  Barry Kirwan,et al.  Development and application of a human error identification tool for air traffic control. , 2002, Applied ergonomics.

[130]  Norman E. Lane,et al.  Improving the Measurement of Team Performance: The TARGETs Methodology , 1994 .

[131]  Pamela McCauley Bell,et al.  A fuzzy linguistic model for the prediction of carpal tunnel syndrome risks in an occupational environment , 1997 .

[132]  Holger Luczak,et al.  Fuzzy modelling of relations between physical weight and perceived heaviness: the effect of size-weight illusion in industrial lifting tasks , 1989 .

[133]  Sidney Dekker,et al.  Drift into Failure: From Hunting Broken Components to Understanding Complex Systems , 2011 .

[134]  B. L. William Wong,et al.  The critical decision method , 2006 .

[135]  W. Chiles EFFECTS OF ELEVATED TEMPERATURES ON PERFORMANCE OF A COMPLEX MENTAL TASK , 1958 .

[136]  Neville A. Stanton,et al.  Distributed Decision Making in Multihelicopter Teams: Case Study of Mission Planning and Execution from a Noncombatant Evacuation Operation Training Scenario , 2010 .

[137]  M. L. Preston,et al.  Twenty-five years of HAZOPs , 1995 .

[138]  J. Crutchfield The calculi of emergence: computation, dynamics and induction , 1994 .

[139]  J. G. Hollands,et al.  Engineering Psychology and Human Performance , 1984 .

[140]  G. Klein,et al.  Cognitive Task Analysis of Teams , 2000 .

[141]  Susan L. Murray,et al.  STAMP – Holistic system safety approach or just another risk model? , 2014 .