Voluntary survey completion among team members: implications of noncompliance and missing data for multilevel research.

We explored whether voluntary survey completion by team members (in aggregate) is predictable from team members' collective evaluations of team-emergent states. In doing so, we reanalyze less-than-complete survey data on 110 teams from a published field study, using so-called traditional and modern missing data techniques to probe the sensitivity of these team-level relationships to data missingness. The multivariate findings revealed that a greater within-team participation rate was indeed related to a higher team-level (mean) score on team mental efficacy (across all four missing-data techniques) and less dispersion among team member judgments about internal cohesion (when the 2 modern methods were used). In addition, results show that a commonly used approach of retaining only those teams with high participation rates produces inflated standardized effect size (i.e., R²) estimates and decreased statistical power. Suggestions include research design considerations and a comprehensive methodology to account for team member data missingness.

[1]  Dmitriy A. Nesterkin,et al.  The Effects of Nonresponse Rates on Group-Level Correlations , 2015 .

[2]  J. Brady,et al.  The Belmont Report. Ethical principles and guidelines for the protection of human subjects of research. , 2014, The Journal of the American College of Dentists.

[3]  L. Offermann,et al.  Too close for comfort? Distinguishing between team intimacy and team cohesion , 2012 .

[4]  Bart de Jong,et al.  Beyond shared perceptions of trust and monitoring in teams: implications of asymmetry and dissensus. , 2012, The Journal of applied psychology.

[5]  Michael S. Cole,et al.  Within-group agreement: On the use (and misuse) of rWG and rWG(J) in leadership research and some best practice guidelines. , 2012 .

[6]  John R. Hollenbeck,et al.  Beyond Team Types and Taxonomies: A Dimensional Scaling Conceptualization for Team Description , 2012 .

[7]  C. Scherbaum,et al.  Identified Employee Surveys: Potential Promise, Perils, and Professional Practice Guidelines , 2011, Industrial and Organizational Psychology.

[8]  Christopher T. Rotolo,et al.  Revisiting the Great Survey Debate: Aren't We Past That Yet? , 2011, Industrial and Organizational Psychology.

[9]  C. Scherbaum,et al.  Identified Employee Surveys: Where Do We Go From Here? , 2011, Industrial and Organizational Psychology.

[10]  P. Rutigliano,et al.  Realizing the Promise and Minimizing the Perils of Identified Surveys: Reports From the Field , 2011, Industrial and Organizational Psychology.

[11]  Justina M. Froelich Identifying the Ethical (Unethical) Undercurrent of Identified Surveys , 2011, Industrial and Organizational Psychology.

[12]  Arthur G. Bedeian,et al.  Dispersion-Composition Models in Multilevel Research , 2011 .

[13]  Pankaj C. Patel,et al.  Demographic faultlines: a meta-analysis of the literature. , 2011, The Journal of applied psychology.

[14]  N. Allen,et al.  Examining workgroup diversity effects: does playing by the (group-retention) rules help or hinder? , 2011, Behavior research methods.

[15]  Noah J. Goldstein,et al.  Managing Normative Influences in Organizations , 2011 .

[16]  Gian Galassi,et al.  Realizing the Promise , 2011 .

[17]  Susan Mohammed,et al.  Marching in-step: Facilitating technological transitions through climate consensus , 2011, Comput. Hum. Behav..

[18]  M. S. Hershcovis,et al.  Interpersonal relationships at work , 2010 .

[19]  Mary E. Zellmer-Bruhn,et al.  Assessing Group-Level Constructs Under Missing Data Conditions: A Monte Carlo Simulation , 2010 .

[20]  David Chan Advances in Analytical Strategies , 2010 .

[21]  R. Millsap,et al.  The SAGE Handbook of Quantitative Methods in Psychology , 2009 .

[22]  Anthony J. Nyberg,et al.  Collective efficacy, group potency, and group performance: meta-analyses of their relationships, and test of a mediation model. , 2009, The Journal of applied psychology.

[23]  Daniel A. Newman,et al.  How Do Missing Data Bias Estimates of Within-Group Agreement? Sensitivity of SD WG, CVWG, rWG(J), rWG(J) * , and ICC to Systematic Nonresponse , 2009 .

[24]  Charles E. Lance,et al.  Statistical and methodological myths and urban legends : doctrine, verity, and fable in the organizational and social sciences , 2009 .

[25]  J. Graham,et al.  Missing data analysis: making it work in the real world. , 2009, Annual review of psychology.

[26]  Robert R. Hirschfeld,et al.  Mental efficacy and physical efficacy at the team level: inputs and outcomes among newly formed action teams. , 2008, The Journal of applied psychology.

[27]  Daniel A. Newman Missing data techniques and low response rates: The role of systematic nonresponse parameters , 2008 .

[28]  James M. LeBreton,et al.  Answers to 20 Questions About Interrater Reliability and Interrater Agreement , 2008 .

[29]  Tammy L. Rapp,et al.  Team Effectiveness 1997-2007: A Review of Recent Advancements and a Glimpse Into the Future , 2008 .

[30]  S. Rogelberg,et al.  Survey Nonrespondents as Bad Soldiers: Examining the Relationship between Organizational Citizenship and Survey Response Behavior , 2007 .

[31]  Michael C. Sturman,et al.  Does the Measure of Dispersion Matter in Multilevel Research? A Comparison of the Relative Performance of Dispersion Indexes , 2007 .

[32]  S. Rogelberg,et al.  Introduction Understanding and Dealing With Organizational Survey Nonresponse , 2007 .

[33]  David J. Stanley,et al.  Assessing the Impact of Nonresponse on Work Group Diversity Effects , 2007 .

[34]  D. Altman,et al.  Missing data , 2007, BMJ : British Medical Journal.

[35]  A. Meade,et al.  Using Indices of Group Agreement in Multilevel Construct Validation , 2007 .

[36]  N. Allen,et al.  Assessing dissimilarity relations under missing data conditions: evidence from computer simulations. , 2007, The Journal of applied psychology.

[37]  D. Dillman Mail and internet surveys: The tailored design method, 2nd ed. , 2007 .

[38]  Louis A. Penner,et al.  The Social Psychology of Prosocial Behavior , 2006 .

[39]  Christopher D. Barr,et al.  “If you treat me right, I reciprocate”: examining the role of exchange in organizational survey response , 2006 .

[40]  Thomas E. Becker Potential Problems in the Statistical Control of Variables in Organizational Research: A Qualitative Analysis With Recommendations , 2005 .

[41]  Kristopher J Preacher,et al.  Use of the extreme groups approach: a critical reexamination and new recommendations. , 2005, Psychological methods.

[42]  M. D. Dunnette Handbook of Industrial and Organizational Psychology , 2005 .

[43]  Thomas A. Timmerman Missing persons in the study of groups , 2005 .

[44]  Craig K. Enders,et al.  Missing Data in Educational Research: A Review of Reporting Practices and Suggestions for Improvement , 2004 .

[45]  William E. Knight,et al.  Profiling active and passive nonrespondents to an organizational survey. , 2003, The Journal of applied psychology.

[46]  Daniel A. Newman Longitudinal Modeling with Randomly and Systematically Missing Data: A Simulation of Ad Hoc, Maximum Likelihood, and Multiple Imputation Techniques , 2003 .

[47]  S. Eckstein The Belmont Report: ethical principles and guidelines for the protection of human subjects of research , 2003 .

[48]  G. Lilien,et al.  Informants in Organizational Marketing Research: Why Use Multiple Informants and how to Aggregate Responses , 2002 .

[49]  D. Cremer,et al.  How do leaders promote cooperation? The effects of charisma and procedural fairness. , 2002 .

[50]  J. Schafer,et al.  Missing data: our view of the state of the art. , 2002, Psychological methods.

[51]  J. Schafer,et al.  A comparison of inclusive and restrictive strategies in modern missing data procedures. , 2001, Psychological methods.

[52]  H. Stern,et al.  The use of multiple imputation for the analysis of missing data. , 2001, Psychological methods.

[53]  John E. Mathieu,et al.  A Temporally Based Framework and Taxonomy of Team Processes , 2001 .

[54]  E. Singer,et al.  Leverage-saliency theory of survey participation: description and an illustration. , 2000, Public opinion quarterly.

[55]  M. Lindell,et al.  Climate quality and climate consensus as mediators of the relationship between organizational antecedents and outcomes. , 2000, The Journal of applied psychology.

[56]  S. Kozlowski,et al.  Multilevel Theory, Research, and Methods in Organizations: Foundations, Extensions, and New Directions , 2000 .

[57]  A. Luong,et al.  Employee attitude surveys: examining the attitudes of noncompliant employees. , 2000, The Journal of applied psychology.

[58]  S. Kozlowski,et al.  Multilevel Theory, Research, a n d M e t h o d s i n Organizations Foundations, Extensions, and New Directions , 2022 .

[59]  Leland Wilkinson,et al.  Statistical Methods in Psychology Journals Guidelines and Explanations , 2005 .

[60]  M P D'Alessandro,et al.  Realizing the promise: delivering pulmonary continuing medical education over the Internet. , 1999, Chest.

[61]  Carl J. Huberty,et al.  Statistical Practices of Educational Researchers: An Analysis of their ANOVA, MANOVA, and ANCOVA Analyses , 1998 .

[62]  D. Chan Functional Relations among Constructs in the Same Content Domain at Different Levels of Analysis: A Typology of Composition Models , 1998 .

[63]  Paul D. Bliese,et al.  Group Consensus and Psychological Well‐Being: A Large Field Study1 , 1998 .

[64]  Steven G. Rogelberg,et al.  Nonresponse to Mailed Surveys , 1998 .

[65]  Roderick J. A. Little,et al.  Modeling the Drop-Out Mechanism in Repeated-Measures Studies , 1995 .

[66]  D. Levy,et al.  Contagion: a theoretical and empirical review and reconceptualization. , 1993, Genetic, social, and general psychology monographs.

[67]  J. Hackman Group influences on individuals in organizations. , 1992 .

[68]  R. Sugden Multiple Imputation for Nonresponse in Surveys , 1988 .

[69]  R. Zajonc SOCIAL FACILITATION. , 1965, Science.

[70]  H. Johnson,et al.  A comparison of 'traditional' and multimedia information systems development practices , 2003, Inf. Softw. Technol..