Investigating naturalistic decision making in a simulated microworld: What questions should we ask?

Computer-simulated microworlds can provide a controlled method for investigating concepts related to naturalistic decision making (NDM). However, the extent to which these tools can be used to generate meaningful outcomes is unknown. The current study used a microworld called Networked Fire Chief (NFC) to explore the range of skills and knowledge acquired as participants gained practice on the program. The complexity of the NFC NDM environment was also explored. Twenty participants each completed 20 equivalent 5-min scenarios on NFC. Interview data, behavioral data and performance scores were collected across the trials. Results confirmed that NFC provides an environment that promotes appropriate perceptual—cognitive processing for NDM. However, performance improved to only a small extent across the 20 trials in four performance areas: speed, accuracy, efficiency and planning. In addition, the number of personal and situational factors to be considered when decision making on NFC was not comparable with real-world NDM environments. Overall, results indicated that the use of microworlds for research should be regulated by an understanding of the limitations of their applicability.

[1]  Roberta Calderwood,et al.  Critical decision method for eliciting knowledge , 1989, IEEE Trans. Syst. Man Cybern..

[2]  J. Shanteau.  Competence in experts: The role of task characteristics , 1992 .

[3]  Henrik Artman.  Situation awareness and co-operation within and between hierarchical units in dynamic decision making , 1999 .

[4]  Ted Nettelbeck,et al.  Investigating the construct validity associated with microworld research: A comparison of performance under different management structures across expert and non-expert naturalistic decision-making groups , 2006 .

[5]  Linda S. Lotto.  Qualitative Data Analysis: A Sourcebook of New Methods , 1986 .

[6]  Clint A. Bowers,et al.  Networked simulations: New paradigms for team performance research , 1995 .

[7]  Jens Rasmussen,et al.  Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[8]  Paul M. Fitts,et al.  Perceptual-Motor Skill Learning1 , 1964 .

[9]  Mary M. Omodei,et al.  The Fire Chief microworld generating program: An illustration of computer-simulated microworlds as an experimental paradigm for studying complex decision-making behavior , 1995 .

[10]  Gary Klein,et al.  Expert decision making , 1999 .

[11]  Gary Klein,et al.  Use of a prediction paradigm to evaluate proficient decision making , 1989 .

[12]  E. Salas,et al.  Establishing the Boundaries of a Paradigm for Decision-Making Research , 1996, Hum. Factors.

[13]  Robert Glaser,et al.  Thoughts on Expertise , 1985 .

[14]  Gary Klein.  Sources of Power , 1998 .

[15]  E. Salas,et al.  Taking stock of naturalistic decision making , 2001 .

[16]  Kimberly A. Neuendorf,et al.  The Content Analysis Guidebook , 2001 .

[17]  B. Brehmer,et al.  Experiments with computer-simulated microworlds: Escaping both the narrow straits of the laboratory and the deep blue sea of the field study , 1993 .

[18]  Gary Klein,et al.  A Recognitional Planning Model , 1999 .

[19]  Mary M. Omodei,et al.  Perceived Difficulty and Motivated Cognitive Effort in a Computer-Simulated Forest Firefighting Task , 1994 .

[20]  Hubert L. Dreyfus,et al.  What computers still can't do - a critique of artificial reason , 1992 .

[21]  Norman E Lane,et al.  FIDELITY AND VALIDITY IN DISTRIBUTED INTERACTIVE SIMULATION: QUESTIONS AND ANSWERS , 1992 .

[22]  Berndt Brehmer,et al.  Reliability and validity of performance measures in microworlds , 2002 .