The Measurement of Team Mental Models: We Have No Shared Schema

This article seeks to promote the advancement of empirical research on team mental models by (a) highlighting the conceptual work that must precede the selection of any measurement tool, (b) delineating measurement standards for group-level cognitions, and (c) evaluating a set of techniques for measuring team mental models. Pathfinder, multidimensional scaling, interactively elicited cognitive mapping, and text-based cognitive mapping are critiqued and compared according to their treatment of content and structure, as well as their psychometric properties. We conclude that these four techniques hold promise for measuring team mental models and illustrate the variability in measurement options. However, careful attention to the research question and research context must precede the selection of any measurement tool.

[1]  J. Mathieu,et al.  The influence of shared mental models on team process and performance. , 2000, The Journal of applied psychology.

[2]  Davide Nicolini.  Comparing Methods for Mapping Organizational Cognition , 1999 .

[3]  Eduardo Salas,et al.  Planning, Shared Mental Models, and Coordinated Performance: An Empirical Link Is Established , 1999, Hum. Factors.

[4]  Michael D. McNeese,et al.  Testing the Effects of Team Processes on Team Member Schema Similarity and Team Performance: Examination of the Team Member Schema Similarity Model , 1998 .

[5]  Kathleen M. Carley.  Extracting team mental models through textual analysis , 1997 .

[6]  Bruce M. Perrin,et al.  Assessing the Stability of Structural Learning Measures , 1997 .

[7]  Eduardo Salas,et al.  Does Overlap in Team Member Knowledge Predict Team Performance? , 1997 .

[8]  K. Kraiger,et al.  Conceptual Development and Empirical Evaluation of Measures of Shared Mental Models as Indicators of Team Effectiveness , 1997 .

[9]  Eduardo Salas,et al.  Measuring Knowledge Organization as a Method for Assessing Learning during Training , 1995, Hum. Factors.

[10]  J. Swan.  Exploring Knowledge and Cognitions in Decisions About Technological Innovation: Mapping Managerial Cognitions , 1995 .

[11]  J. Walsh.  Managerial and Organizational Cognition: Notes from a Trip Down Memory Lane , 1995 .

[12]  N. Shadbolt,et al.  Eliciting Knowledge from Experts: A Methodological Analysis , 1995 .

[13]  Lívia Markíczy,et al.  A Method for Eliciting and Comparing Causal Maps , 1995 .

[14]  D. P. Baker,et al.  Measuring the Importance of Teamwork: The Reliability and Validity of Job/Task Analysis Indices for Team-Training Design , 1994 .

[15]  Mauri Laukkanen,et al.  Comparative Cause Mapping of Organizational Cognitions , 1994 .

[16]  G. Johnson,et al.  Ceos' cognitive maps and the scope of the organization , 1994 .

[17]  L. Robin Keller,et al.  The Effect of Inference Order and Experience-Related Knowledge on Diagnostic Conjunction Probabilities , 1994 .

[18]  Kathleen M. Carley.  Extracting culture through textual analysis , 1994 .

[19]  Gregory Ashby,et al.  On the Dangers of Averaging Across Subjects When Using Multidimensional Scaling or the Similarity-Choice Model , 1994 .

[20]  S. Mohammed,et al.  Team Mental Model: Construct or Metaphor? , 1994 .

[21]  Joseph F. Porac,et al.  Advances in managerial cognition and organizational information processing , 1994 .

[22]  R. Hall,et al.  Members of great teams think alike: A model of team effectiveness and schema similarity among team members. , 1994 .

[23]  Susan C. Schneider,et al.  Cognition in Organizational Analysis: Who's Minding the Store? , 1993 .

[24]  G. Matthew Bonham,et al.  Cognitive mapping as a technique for supporting international negotiation , 1993 .

[25]  J. R. Larson,et al.  Groups as problem‐solving units: Toward a new meaning of social cognition , 1993 .

[26]  David H. Jonassen,et al.  Structural Knowledge: Techniques for Representing, Conveying, and Acquiring Structural Knowledge , 1993 .

[27]  Tony Simons,et al.  Speech Patterns and the Concept of Utility in Cognitive Maps: The Case of Integrative Bargaining , 1993 .

[28]  P. Barr,et al.  Cognitive change, strategic action and organizational renewal , 1993 .

[29]  E. Salas,et al.  Shared mental models in expert team decision making. , 1993 .

[30]  David S. Glosser,et al.  A Multidimensional Approach , 1993 .

[31]  E. Salas,et al.  Team decision making in complex environments. , 1993 .

[32]  Kathleen M. Carley.  Coding Choices for Textual Analysis: A Comparison of Content Analysis and Map Analysis , 1993 .

[33]  William B. Rouse,et al.  Human-Technology Interaction in Complex Systems , 1993 .

[34]  Nancy J. Cooke,et al.  Eliciting Semantic Relations for Empirically Derived Networks , 1992, Int. J. Man Mach. Stud..

[35]  Andrew Wirth,et al.  Measuring Differences Between Cognitive Maps , 1992 .

[36]  William B. Rouse,et al.  The role of mental models in team performance in complex systems , 1992, IEEE Trans. Syst. Man Cybern..

[37]  C. Eden,et al.  The analysis of cause maps , 1992 .

[38]  C. Eden.  ON THE NATURE OF COGNITIVE MAPS , 1992 .

[39]  Sylvia M. Brown,et al.  Cognitive mapping and repertory grids for qualitative survey research: some , 1992 .

[40]  K. Langfield-Smith.  EXPLORING THE NEED FOR A SHARED COGNITIVE MAP , 1992 .

[41]  Nancy J. Cooke,et al.  Predicting judgment time from measures of psychological proximity. , 1992 .

[42]  Michel G. Bougon,et al.  CONGREGATE COGNITIVE MAPS: A UNIFIED DYNAMIC THEORY OF ORGANIZATION AND STRATEGY , 1992 .

[43]  Michel Audet,et al.  MAPPING OF AN IDIOSYNCRATIC SCHEMA , 1992 .

[44]  Anne Sigismund Huff,et al.  MAPS FOR MANAGERS: WHERE ARE WE? WHERE DO WE GO FROM HERE? , 1992 .

[45]  Nancy J. Cooke,et al.  Network and Multidimensional Representations of the Declarative Knowledge of Human-Computer Interface Design Experts , 1992, Int. J. Man Mach. Stud..

[46]  Kathleen M. Carley,et al.  Extracting, Representing, and Analyzing Mental Models , 1992 .

[47]  L. E. Klopfer,et al.  Multitechnique-Multimethod Matrix Methodology for Construct Validation and its Application for the Construct of Cognitive Structure , 1992 .

[48]  Tim Smithin,et al.  Mapping Strategic Thought , 1991 .

[49]  R. Nelson,et al.  Cause Maps and Social Network Analysis in Organizational Diagnosis , 1991 .

[50]  Peder J. Johnson,et al.  Assessing Structural Knowledge. , 1991 .

[51]  Peter H. Reingen,et al.  Sociocognitive Analysis of Group Decision Making among Consumers , 1990 .

[52]  J. Rentsch.  Climate and culture : interaction and qualitative differences in organizational meanings , 1990 .

[53]  J. Novak.  Concept mapping: A useful tool for science education , 1990 .

[54]  Roger W. Schvaneveldt,et al.  Proximities, networks, and schemata , 1990 .

[55]  Timothy E. Goldsmith,et al.  A structural assessment of classroom learning , 1990 .

[56]  Francis T. Durso,et al.  Graphs in the social and psychological sciences: empirical contributions of pathfinder , 1990 .

[57]  R. Branaghan.  Pathfinder networks and multidimensional spaces: relative strengths in representing strong associates , 1990 .

[58]  Chris Esposito,et al.  Fuzzy PFNETs: coping with variability in proximity data , 1990 .

[59]  T. Goldsmith,et al.  Assessing structural similarity of graphs , 1990 .

[60]  Nancy J. Cooke,et al.  Using pathfinder as knowledge elicitation tool: link interpretation , 1990 .

[61]  John G. Gammack,et al.  Expert conceptual structure: the stability of pathfinder representations , 1990 .

[62]  H. Thomas,et al.  Taxonomic Mental Models in Competitor Definition , 1990 .

[63]  J. R. Larson,et al.  Research strategies and tactics in industrial and organizational psychology. , 1990 .

[64]  Mauri Laukkanen,et al.  Describing management cognition: The cause mapping approach , 1990 .

[65]  Liam Fahey,et al.  LINKING CHANGES IN REVEALED CAUSAL MAPS AND ENVIRONMENTAL CHANGE: AN EMPIRICAL STUDY , 1989 .

[66]  C. Stubbart.  MANAGERIAL COGNITION: A MISSING LINK IN STRATEGIC MANAGEMENT RESEARCH , 1989 .

[67]  C. Marlene Fiol,et al.  A Semiotic Analysis of Corporate Language: Organizational Boundaries and Joint Venturing , 1989 .

[68]  Eduardo Salas,et al.  Toward a Definition of Teamwork: An Analysis of Critical Team Behaviors , 1989 .

[69]  J. Ford,et al.  Process tracing methods: Contributions, problems, and neglected research questions , 1989 .

[70]  Francis T. Durso,et al.  Network Structures in Proximity Data , 1989 .

[71]  James P. Walsh,et al.  Selectivity and selective perception: An investigation of managers''belief structures and informatio , 1988 .

[72]  James P. Walsh,et al.  Negotiated belief structures and decision performance: An empirical investigation , 1988 .

[73]  Nancy J. Cooke,et al.  Effects of Computer Programming Experience on Network Representations of Abstract Programming Concepts , 1988, Int. J. Man Mach. Stud..

[74]  James E. McDonald,et al.  The Application of Psychological Scaling Techniques to Knowledge Elicitation for Knowledge-Based Systems , 1987, Int. J. Man Mach. Stud..

[75]  Francis T. Durso,et al.  Recall and Measures of Memory Organization , 1986 .

[76]  J. Walsh,et al.  The Role of Negotiated Belief Structures in Strategy Making , 1986 .

[77]  A. Donnellon,et al.  Communication, Meaning, and Organized Action. , 1986 .

[78]  Roger W. Schvaneveldt,et al.  Measuring the Structure of Expertise , 1985, Int. J. Man Mach. Stud..

[79]  Arkalgud Ramaprasad,et al.  A computerized interactive technique for mapping influence diagrams (MIND) , 1985 .

[80]  Barbara Gray,et al.  Organizations as Constructions and Destructions of Meaning , 1985 .

[81]  J. Bartunek.  Changing Interpretive Schemes and Organizational Restructuring: The Example of a Religious Order , 1984 .

[82]  R. Hall.  The Natural Logic of Management Policy Making: Its Implications for the Survival of an Organization , 1984 .

[83]  W. Harvey Hegarty,et al.  Decision Makers' Beliefs About the Causes and Effects of Structure: An Exploratory Study , 1984 .

[84]  Robert Chan,et al.  Demonstration of Multi-Dimensional Scaling as an Evaluation Technique: Evaluation of the Vertical Team Concept in an Evaluation Training Programr , 1984 .

[85]  W. Gingerich.  Methodological Observations on Applied Behavioral Science , 1983, The Journal of applied behavioral science.

[86]  Rochel Gelman,et al.  The Child's Representation of Number: A Multidimensional Scaling Analysis. , 1983 .

[87]  Edward J. Shoben,et al.  Applications of Multidimensional Scaling in Cognitive Psychology , 1983 .

[88]  L. Jones.  Multidimensional Models of Social Perception, Cognition, and Behavior , 1983 .

[89]  Robert F. Stanners,et al.  The Assessment and Modification of Concept Interrelationships , 1983 .

[90]  J. Rounds,et al.  Multidimensional Scaling Research in Vocational Psychology , 1983 .

[91]  Barton A. Weitz,et al.  Attributions in the Board Room: Causal Reasoning in Corporate Annual Reports , 1983 .

[92]  Judith R. Gordon,et al.  Developing “Complicated” Understanding in Administrators , 1983 .

[93]  G. Diekhoff.  Testing Through Relationship Judgments. , 1983 .

[94]  Joseph P. Forgas,et al.  Episode cognition and personality: A multidimensional analysis , 1983 .

[95]  Rosedith Sitgreaves Bowker.  Measurement Theory for the Behavioral Sciences , 1983 .

[96]  Lucy Pollard-Gott,et al.  Emergence of thematic concepts in repeated listening to music , 1983, Cognitive Psychology.

[97]  A. Carrón,et al.  Cohesiveness in Sport Groups: Interpretations ani Considerations , 1982 .

[98]  E. Hirschman,et al.  Free-Response and Card-Sort Techniques for Assessing Cognitive Content: Two Studies concerning Their Stability, Validity, and Utility , 1982 .

[99]  G. W. Hill.  Group versus individual performance: are n + 1 heads better than one?" psychological bulletin , 1982 .

[100]  J. Diffenbach.  Influence diagrams for complex strategic issues , 1982 .

[101]  Forrest W. Young,et al.  Introduction to Multidimensional Scaling: Theory, Methods, and Applications , 1981 .

[102]  J. Forgas.  Social episodes and group milieu: A study in social cognition , 1981 .

[103]  C. Eden,et al.  THE INTERSUBJECTIVITY OF ISSUES AND ISSUES OF INTERSUBJECTIVITY , 1981 .

[104]  Myron Wish,et al.  Perceived Dimensions of Interpersonal Relations , 1981 .

[105]  Philip E. Tetlock,et al.  A Cognitive Analysis of Japan's 1941 Decision for War , 1980 .

[106]  R. Billings,et al.  DIMENSIONS OF WORK OUTCOMES: A MULTIDIMENSIONAL SCALING APPROACH , 1980 .

[107]  Roger I. Hall,et al.  Influence Diagrams and Organizational Power. , 1980 .

[108]  Edward G. Carmines,et al.  Reliability and Validity Assessment , 1979 .

[109]  K. O’grady,et al.  A Multidimensional Scaling Analysis Of Sex Guilt. , 1979, Multivariate behavioral research.

[110]  OlgaS. Burn,et al.  Developing an interdisciplinary science of organizations , 1979 .

[111]  F Dragsgow,et al.  Multidimensional Scaling Of Derived Dissimilarities. , 1979, Multivariate behavioral research.

[112]  R. MacCallum,et al.  Recovery of structure in incomplete data by alscal , 1979 .

[113]  J. F. Voss,et al.  Prose Representation: A Multidimensional Scaling Approach. , 1979, Multivariate behavioral research.

[114]  J. Forgas.  Social episodes and social structure in an academic setting: The social environment of an intact group , 1978 .

[115]  Gregg T. Vesonder,et al.  On the representation of prose: New dimensions , 1978 .

[116]  P. Preece.  Three-Year Stability of Certain Word-Association Indices , 1978 .

[117]  M. Bougon.  Cognition in Organizations: An Analysis of the Utrecht Jazz Orchestra. , 1977 .

[118]  R. MacCallum,et al.  A monte carlo investigation of recovery of structure by alscal , 1977 .

[119]  R. MacCallum,et al.  Effects of conditionality on INDSCAL and ALSCAL weights , 1977 .

[120]  D. Howard,et al.  A Multidimensional Scaling Analysis of the Development of Animal Names. , 1977 .

[121]  Lawrence E. Jones,et al.  Individual differences in perceptions and preferences for political candidates , 1977 .

[122]  P. Preece.  Science Concepts in Semantic Space--A Multidimensional Scaling Study. , 1976 .

[123]  Peter H. Schönemann,et al.  Alternative measures of fit for the Schönemann-carroll matrix fitting algorithm , 1974 .

[124]  Paul E. Green,et al.  Applied multidimensional scaling : a comparison of approaches and algorithms , 1972 .

[125]  P. Schönemann,et al.  Fitting one matrix to another under choice of a central dilation and a rigid motion , 1970 .

[126]  P. S. Vivekananthan,et al.  A multidimensional approach to the structure of personality impressions. , 1968, Journal of personality and social psychology.

[127]  Ernest J. McCormick,et al.  Paired comparison ratings: 2. The reliability of ratings based on partial pairings. , 1952 .

[128]  Ernest J. McCormick,et al.  Paired comparison ratings. I. The effect of ratings of reductions in the number of pairs. , 1952 .