Using situation awareness measures to characterize mental models in an inductive reasoning task

Abstract The objective of this study was to develop and validate an empirical method of situation awareness (SA) assessment for characterizing mental models in an inductive reasoning task. We conducted an experiment in which participants watched videos of two detective stories while simultaneously playing a simple shooting game. Participant SA was measured during the trial and post-trial measures included concept mapping and knowledge tests on the stories. A fuzzy inference model was developed to classify SA measurement outcomes according to patterns that matched with hypothesized types of mental models. The model type identification, based on fuzzy inference, revealed utility for predicting knowledge test performance. The fuzzy model outputs and SA measures were found to be independent from (or complementary to) concept map measures. The results provided empirical evidence of the utility of SA measures for inferring forms of mental models and predicting knowledge of an inductive reasoning task, independent of an existing assessment approach (concept mapping).

[1]  M. Angela Sasse,et al.  Eliciting and describing users' models of computer systems , 1997 .

[2]  Mica R. Endsley,et al.  Toward a Theory of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[3]  John Patrick,et al.  A task-oriented perspective of situation awareness , 2004 .

[4]  Cleotilde Gonzalez,et al.  ACognitive Approach to Game Usability and Design: Mental Model Development in Novice Real-Time Strategy Gamers , 2006, Cyberpsychology Behav. Soc. Netw..

[5]  Nancy J. Cooke,et al.  Network and Multidimensional Representations of the Declarative Knowledge of Human-Computer Interface Design Experts , 1992, Int. J. Man Mach. Stud..

[6]  Ian D. Graham,et al.  Book review: Applications of fuzzy set theory in human factors , 1987 .

[7]  David B. Kaber,et al.  Effects of physical workload on cognitive task performance and situation awareness , 2008 .

[8]  Lee A. Freeman,et al.  The power and benefits of concept mapping: measuring use, usefulness, ease of use, and satisfaction , 2004 .

[9]  Michelle Moloney,et al.  Knowledge, expectations, and inductive reasoning within conceptual hierarchies , 2004, Cognition.

[10]  Bernard C. Jiang,et al.  Fuzzy Modeling of Combined Manual Materials-Handling Capacities , 1986 .

[11]  P. Johnson-Laird,et al.  Mental Models: Towards a Cognitive Science of Language, Inference, and Consciousness , 1985 .

[12]  Christine L. Borgman.  The User's Mental Model of an Information Retrieval System: An Experiment on a Prototype Online Catalog , 1986, Int. J. Man Mach. Stud..

[13]  Roger W. Schvaneveldt,et al.  Pathfinder associative networks: studies in knowledge organization , 1990 .

[14]  D. Gopher,et al.  Attention and performance XVII: Cognitive regulation of performance: Interaction of theory and application. , 1999 .

[15]  C A Nelson,et al.  Learning to Learn , 2017, Encyclopedia of Machine Learning and Data Mining.

[16]  Stephen J. Payne,et al.  CHAPTER 6 – Users' Mental Models: The Very Ideas , 2003 .

[17]  David Greathead,et al.  When mental models go wrong: co-occurrences in dynamic, critical systems , 2004, Int. J. Hum. Comput. Stud..

[18]  M. Endsley.  Situation Models: An Avenue to the Modeling of Mental Models , 2000 .

[19]  Neville Moray.  Intelligent Aids, Mental Models, and the Theory of Machines , 1987, Int. J. Man Mach. Stud..

[20]  Paul Thagard,et al.  Induction: Processes Of Inference , 1989 .

[21]  David B. Kaber,et al.  Characterisation of mental models in a virtual reality-based multitasking scenario using measures of situation awareness , 2010 .

[22]  Daniel J. Garland,et al.  Situation Awareness Analysis and Measurement , 2009 .

[23]  Donald A. Norman,et al.  Some observations on mental models , 1987 .

[24]  Mica R. Endsley,et al.  Providing understanding of the behavior of feedforward neural networks , 1997, IEEE Trans. Syst. Man Cybern. Part B.

[25]  Colin G. Drury,et al.  An Application of Fuzzy Set Theory for Evaluation of Human Performance on an Inspection Task , 1986 .

[26]  James D. Hollan,et al.  Human Reasoning About a Simple Physical System , 2014 .

[27]  David B. Kaber,et al.  A fuzzy system hazard analysis approach for human-in-the-loop systems , 2019 .

[28]  Janice Langan-Fox,et al.  Team Mental Models: Techniques, Methods, and Analytic Approaches , 2000, Hum. Factors.

[29]  Yvonne Rogers,et al.  Models in the mind : theory, perspective and application , 1992 .

[30]  David B. Kaber,et al.  Situation awareness and workload in driving while using adaptive cruise control and a cell phone , 2005 .

[31]  John R. Wilson,et al.  Mental Models: Theory and Application in Human Factors , 1989 .

[32]  Thomas P. Moran,et al.  The Command Language Grammar: A Representation for the User Interface of Interactive Computer Systems , 1981, Int. J. Man Mach. Stud..

[33]  Florian Jentsch,et al.  Mental Model Assessments: Is There Convergence Among Different Methods? , 2001 .

[34]  Nancy Staggers,et al.  Mental Models: Concepts for Human-Computer Interaction Research , 1993, Int. J. Man Mach. Stud..

[35]  N. M. Morris,et al.  On Looking into the Black Box: Prospects and Limits in the Search for Mental Models , 1986 .

[36]  Wendy A. Kellogg,et al.  Evaluating user and system models: applying scaling techniques to problems in human-computer interaction , 1986, CHI '87.

[37]  Henri Prade,et al.  Dealing with the Vagueness of Natural Languages in Man-Machine Communication , 1986 .

[38]  Mica R. Endsley,et al.  Theoretical Underpinnings of Situation Awareness, A Critical Review , 2000 .

[39]  Guy H. Walker,et al.  What Really Is Going on? Review, Critique and Extension of Situation Awareness Theory , 2007, HCI.

[40]  Rocky Ross,et al.  Mental models , 2004, SIGA.

[41]  Penelope M. Sanderson.  Knowledge acquisition and fault diagnosis: experiments with PLAULT , 1990, IEEE Trans. Syst. Man Cybern..

[42]  David B. Kaber,et al.  The effects of level of automation and adaptive automation on human performance, situation awareness and workload in a dynamic control task , 2004 .