Measuring Cognition in Teams

Objective: The purpose of this article is twofold: to provide a critical cross-domain evaluation of team cognition measurement options and to provide novice researchers with practical guidance when selecting a measurement method. Background: A vast selection of measurement approaches exist for measuring team cognition constructs including team mental models, transactive memory systems, team situation awareness, strategic consensus, and cognitive processes. Methods: Empirical studies and theoretical articles were reviewed to identify all of the existing approaches for measuring team cognition. These approaches were evaluated based on theoretical perspective assumed, constructs studied, resources required, level of obtrusiveness, internal consistency reliability, and predictive validity. Results: The evaluations suggest that all existing methods are viable options from the point of view of reliability and validity, and that there are potential opportunities for cross-domain use. For example, methods traditionally used only to measure mental models may be useful for examining transactive memory and situation awareness. The selection of team cognition measures requires researchers to answer several key questions regarding the theoretical nature of team cognition and the practical feasibility of each method. Conclusions: We provide novice researchers with guidance regarding how to begin the search for a team cognition measure and suggest several new ideas regarding future measurement research. Applications: We provide (1) a broad overview and evaluation of existing team cognition measurement methods, (2) suggestions for new uses of those methods across research domains, and (3) critical guidance for novice researchers looking to measure team cognition.

[1]  Nancy J. Cooke,et al.  Team cognition in experienced command-and-control teams. , 2007, Journal of experimental psychology. Applied.

[2]  Cg Christel Rutte,et al.  Meeting deadlines in work groups: Implicit and explicit mechanisms , 2006 .

[3]  A P Banks,et al.  Running shared mental models as a distributed cognitive process. , 2000, British journal of psychology.

[4]  Chi-Cheng Huang.  Knowledge sharing and group cohesiveness on performance: An empirical study of technology R&D teams in Taiwan , 2009 .

[5]  Nancy J. Cooke,et al.  Measuring mental models: Choosing the right tools for the job , 1995 .

[6]  Quinetta M. Roberson,et al.  Justice in teams : The activation and role of sensemaking in the emergence of justice climates , 2006 .

[7]  Nancy J. Cooke,et al.  On Teams, Teamwork, and Team Performance: Discoveries and Developments , 2008, Hum. Factors.

[8]  Lelyn D. Saner,et al.  Measuring and Predicting Shared Situation Awareness in Teams , 2009 .

[9]  Gunnar Hauland.  Measuring Individual and Team Situation Awareness During Planning Tasks in Training of En Route Air Traffic Control , 2008 .

[10]  Linda R. Hare,et al.  Effectiveness and Efficiency in Small Academic Peer Groups , 2000 .

[11]  Erich C. Dierdorff,et al.  Facet personality and surface-level diversity as team mental model antecedents: implications for implicit coordination. , 2012, The Journal of applied psychology.

[12]  Neville A. Stanton,et al.  Is SA shared or distributed in team work? An exploratory study in an intelligence analysis task , 2011 .

[13]  Kimberly A. Smith-Jentsch,et al.  Measuring team-related cognition: The devil is in the details. , 2008 .

[14]  Balagopal Vissa,et al.  Leveraging Ties: The Contingent Value of Entrepreneurial Teams' External Advice Networks on Indian Software Venture Performance , 2009 .

[15]  J Patrick,et al.  Observational assessment of situation awareness, team differences and training implications , 2006, Ergonomics.

[16]  S. Borgatti,et al.  The Network Paradigm in Organizational Research: A Review and Typology , 2003 .

[17]  Mica R. Endsley,et al.  Design and Evaluation for Situation Awareness Enhancement , 1988 .

[18]  J Bourbousson,et al.  Description of dynamic shared knowledge: an exploratory study during a competitive team sports interaction , 2011, Ergonomics.

[19]  J. Mathieu,et al.  Scaling the quality of teammates' mental models: equifinality and normative comparisons , 2005 .

[20]  S. Mohammed,et al.  The Measurement of Team Mental Models: We Have No Shared Schema , 2000 .

[21]  Michael D. Ensley,et al.  A reciprocal and longitudinal investigation of the innovation process: the central role of shared vision in product and process innovation teams (PPITs) , 2004 .

[22]  Eduardo Salas,et al.  Analyzing Communication Sequences for Team Training Needs Assessment , 1998, Hum. Factors.

[23]  Patrick F. Reidy.  An Introduction to Latent Semantic Analysis , 2009 .

[24]  Fraser J. M. Reid,et al.  Cognitive Entrainment in Engineering Design Teams , 2000 .

[25]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[26]  Nancy J. Cooke,et al.  Measuring Team Knowledge , 2000, Hum. Factors.

[27]  L. Sproull,et al.  Coordinating Expertise in Software Development Teams , 2000 .

[28]  Nancy J. Cooke,et al.  Advances in Measuring Team Cognition , 2003 .

[29]  E. Salas,et al.  Shared Expectations and Implicit Coordination in Tennis Doubles Teams , 2010 .

[30]  G. Underwood,et al.  The Role of Eye Movements in Reading: Some Limitations of the Eye-Mind Assumption , 1992 .

[31]  Ajay Mehra,et al.  Top Management-Team Diversity and Firm Performance: Examining the Role of Cognitions , 2000 .

[32]  Jane M. Howell,et al.  Leading Military Teams to Think and Feel: Exploring the Relations Between Leadership, Soldiers' Cognitive and Affective Processes, and Team Effectiveness , 2009 .

[33]  Gwendolyn E. Campbell,et al.  Measuring teamwork mental models to support training needs assessment, development, and evaluation: two empirical studies† , 2001 .

[34]  P. Kirschner,et al.  Social and Cognitive Factors Driving Teamwork in Collaborative Learning Environments , 2006 .

[35]  Nancy J. Cooke,et al.  Measuring Patterns in Team Interaction Sequences Using a Discrete Recurrence Approach , 2012, Hum. Factors.

[36]  S. Mohammed,et al.  Metaphor No More: A 15-Year Review of the Team Mental Model Construct , 2010 .

[37]  S. Mohammed,et al.  Team Mental Model: Construct or Metaphor? , 1994 .

[38]  Peter W. Foltz,et al.  An introduction to latent semantic analysis , 1998 .

[39]  Guy H. Walker,et al.  What really is going on? Review of situation awareness models for individuals and teams , 2008 .

[40]  S. Floyd,et al.  Managing strategic consensus: the foundation of effective implementation , 1992 .

[41]  Richard I. Cook,et al.  Sensemaking, safety, and cooperative work in the intensive care unit , 2007, Cognition, Technology & Work.

[42]  Nancy J. Cooke,et al.  Procedural Network Representations of Sequential Data , 1996, Hum. Comput. Interact..

[43]  J. Mathieu,et al.  Performance implications of leader briefings and team-interaction training for team adaptation to novel environments. , 2000, The Journal of applied psychology.

[44]  R. Moreland,et al.  Group Versus Individual Training and Group Performance: The Mediating Role of Transactive Memory , 1995 .

[45]  R. Moreland,et al.  Exploring the Performance Benefits of Group Training: Transactive Memory or Improved Communication? , 2000 .

[46]  Peter W. Foltz,et al.  Automated Communication Analysis of Teams , 2008 .

[47]  Colin F. Mackenzie,et al.  Adaptation of Team Communication Patterns: Exploring the Effects of Leadership at a Distance: Task Urgency, and Shared Team Experience , 2007 .

[48]  Mien Segers,et al.  Reconsidering group cognition: From conceptual confusion to a boundary area between cognitive and socio-cultural perspectives? , 2007 .

[49]  Walter C. Borman,et al.  Job behavior, performance, and effectiveness. , 1991 .

[50]  M. D. Dunnette.  Handbook of Industrial and Organizational Psychology , 2005 .

[51]  Luis L. Martins,et al.  Cognitive diversity and team performance: the roles of team mental models and information processing mechanisms , 2010 .

[52]  N A Stanton,et al.  Distributed situation awareness in dynamic systems: theoretical development and application of an ergonomics methodology , 2006, Ergonomics.

[53]  Devaki Rau,et al.  Top management team transactive memory, information gathering, and perceptual accuracy , 2006 .

[54]  D. Wegner.  Transactive Memory: A Contemporary Analysis of the Group Mind , 1987 .

[55]  Ramón Valle-Cabrera,et al.  The relationship between top management teams and innovative capacity in companies , 2005 .

[56]  R. Clariana,et al.  A Computer-Based Approach for Deriving and Measuring Individual and Team Knowledge Structure from Essay Questions , 2007 .

[57]  Vesa Peltokorpi,et al.  Transactive Memory Systems , 2008 .

[58]  Cleotilde Gonzalez,et al.  Shared Mental Models in Military Command and Control Organizations: Effect of Social Network Distance , 2004 .

[59]  Laurie L. Levesque,et al.  Cognitive divergence and shared mental models in software development project teams , 2001 .

[60]  Nancy J. Cooke,et al.  Measuring Team Knowledge: A Window to the Cognitive Underpinnings of Team Performance , 2003 .

[61]  Aleksander P. J. Ellis.  System Breakdown: The Role of Mental Models and Transactive Memory in the Relationship between Acute Stress and Team Performance , 2006 .

[62]  Gervase R. Bushe,et al.  Group Development and Team Effectiveness , 2007 .

[63]  Yanfeng Zheng.  Unlocking Founding Team Prior Shared Experience: A Transactive Memory System Perspective , 2012 .

[64]  H. Chou,et al.  Team cognition, collective efficacy, and performance in strategic decision-making teams , 2012 .

[65]  B. Cheng,et al.  Shared work values and team member effectiveness: The mediation of trustfulness and trustworthiness , 2008 .

[66]  Norbert K. Semmer,et al.  Explicit Reasoning, Confirmation Bias, and Illusory Transactive Memory , 2009 .

[67]  James D. Herbsleb,et al.  Team Knowledge and Coordination in Geographically Distributed Software Development , 2007, J. Manag. Inf. Syst..

[68]  Eduardo Salas,et al.  Measurement of Team Situation Awareness in Low Experience Level Aviators , 2007 .

[69]  V. Hinsz.  Metacognition and mental models in groups: An illustration with metamemory of group recognition memory. , 2004 .

[70]  Marshall Scott Poole,et al.  Research methods for studying groups and teams : a guide to approaches, tools, and technologies , 2012 .

[71]  Eva Jensen,et al.  Sensemaking in military planning: a methodological study of command teams , 2009, Cognition, Technology & Work.

[72]  Rashaad E. T. Jones,et al.  The development of an emergency crisis management simulation to assess the impact a fuzzy cognitive map decision-aid has on team cognition and team decision-making , 2006 .

[73]  Eduardo Salas,et al.  Facilitating Knowledge Building in Teams: Can a New Team Training Strategy Help? , 2010 .

[74]  Mary J. Waller,et al.  Team Cognition and Adaptability in Dynamic Settings: A review of Pertinent Work , 2010 .

[75]  Eduardo Salas,et al.  TPL—KATS-card sort: A tool for assessing structural knowledge , 2003, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[76]  D. Hambrick.  Strategic Awareness Within Top Management Teams. , 1980 .

[77]  S. Floyd,et al.  The Lack of Consensus About Strategic Consensus: Advancing Theory and Research , 2005 .

[78]  Leslie A. DeChurch,et al.  Mental model metrics and team adaptability: A multi-facet multi-method examination , 2010 .

[79]  Petra Badke-Schaub,et al.  Shared mental models—linking team cognition and performance , 2007 .

[80]  Roger W. Schvaneveldt,et al.  Pathfinder associative networks: studies in knowledge organization , 1990 .

[81]  R. Bakeman,et al.  Observing interaction: An introduction to sequential analysis, 2nd ed. , 1997 .

[82]  Kathleen M. Carley.  Extracting team mental models through textual analysis , 1997 .

[83]  Yuqing Ren,et al.  Transactive Memory Systems 1985–2010: An Integrative Framework of Key Dimensions, Antecedents, and Consequences , 2011 .

[84]  Ananth Chiravuri,et al.  Cognitive Conflict and Consensus Generation in Virtual Teams During Knowledge Capture: Comparative Effectiveness of Techniques , 2011, J. Manag. Inf. Syst..

[85]  Nancy J. Cooke,et al.  Measuring Team Knowledge During Skill Acquisition of a Complex Task , 2001 .

[86]  Neil Anderson,et al.  Working hard and working smart: motivation and ability during typical and maximum performance. , 2007, The Journal of applied psychology.

[87]  John R. Austin.  Transactive memory in organizational groups: the effects of content, consensus, specialization, and accuracy on group performance. , 2003, The Journal of applied psychology.

[88]  Leslie A. DeChurch,et al.  The cognitive underpinnings of effective teamwork: a meta-analysis. , 2010, The Journal of applied psychology.

[89]  Kyle Lewis.  Measuring transactive memory systems in the field: scale development and validation. , 2003, The Journal of applied psychology.

[90]  E. Salas,et al.  Shared mental models in expert team decision making. , 1993 .

[91]  Eduardo Salas,et al.  Team Effectiveness in Complex Organizations: Cross-Disciplinary Perspectives and Approaches , 2008 .

[92]  Anne Miller,et al.  Coordination and Continuity of Intensive Care Unit Patient Care , 2009, Hum. Factors.

[93]  C. Burke,et al.  The impact of cross-training on team effectiveness. , 2002, The Journal of applied psychology.

[94]  Debra L. O'Connor,et al.  Measuring Sharedness of Team-Related Knowledge: Design and Validation of a Shared Mental Model Instrument , 2007 .

[95]  Janice Langan-Fox,et al.  Team Mental Models: Techniques, Methods, and Analytic Approaches , 2000, Hum. Factors.

[96]  Mohamed Louadi.  Knowledge heterogeneity and social network analysis – Towards conceptual and measurement clarifications , 2008 .

[97]  Nancy J. Cooke,et al.  Interactive Team Cognition , 2013, Cogn. Sci..

[98]  Danko Nikolić,et al.  SITUATION AWARENESS AS A PREDICTOR OF PERFORMANCE IN EN ROUTE AIR TRAFFIC CONTROLLERS , 1998 .

[99]  Nancy J. Cooke,et al.  Changes in team cognition after a retention interval: the benefits of mixing it up. , 2011, Journal of experimental psychology. Applied.

[100]  Philip Smith.  Roger Bakeman John M. Gottman , 1987, Animal Behaviour.

[101]  Eric Anthony Day,et al.  Relationships among team ability composition, team mental models, and team performance. , 2006, The Journal of applied psychology.

[102]  J. Mathieu,et al.  Investigating linear and interactive effects of shared mental models on safety and efficiency in a field setting. , 2005, The Journal of applied psychology.

[103]  M. Endsley,et al.  Individual Differences in Pilot Situation Awareness , 1994 .

[104]  Nancy J. Cooke,et al.  Measuring team situation awareness in decentralized command and control environments , 2006, Ergonomics.

[105]  Mica R. Endsley,et al.  Measuring Shared and Team Situation Awareness in the Army's Future Objective Force , 2003 .

[106]  Norman E. Lane,et al.  Improving the Measurement of Team Performance: The TARGETs Methodology , 1994 .

[107]  Nancy J. Cooke,et al.  Interaction-Based Measures of Cognitive Systems , 2009 .

[108]  Nancy J. Cooke,et al.  An Ecological Perspective on Team Cognition , 2004 .

[109]  Janice Langan-Fox,et al.  Mental models, team mental models, and performance: Process, development, and future directions , 2004 .

[110]  Neville Stanton,et al.  Situation awareness measurement: a review of applicability for C4i environments. , 2006, Applied ergonomics.