Missing Data in Multiple Item Scales: A Monte Carlo Analysis of Missing Data Techniques

Researchers in many fields use multiple item scales to measure important variables such as attitudes and personality traits, but find that some respondents failed to complete certain items. Past missing data research focuses on missing entire instruments, and is of limited help because there are few variables to help impute missing scores and the variables are often not highly related to each other. Multiple item scales offer the unique opportunity to impute missing values from other correlated items designed to measure the same construct. A Monte Carlo analysis was conducted to compare several missing data techniques. The techniques included listwise deletion, regression imputation, hot-deck imputation, and two forms of mean substitution. Results suggest that regression imputation and substituting the mean response of a person to other items on a scale are very promising approaches. Furthermore, the imputation techniques often outperformed listwise deletion.

[1]  S. F. Buck.  A Method of Estimation of Missing Values in Multivariate Data Suitable for Use with an Electronic Computer , 1960 .

[2]  D. Weiss,et al.  Manual for the Minnesota Satisfaction Questionnaire. , 1967 .

[3]  S. Lirtzman,et al.  Role Conflict and Ambiguity in Complex Organizations. , 1970 .

[4]  H. Mirels,et al.  The Protestant ethic as a personality variable. , 1971, Journal of consulting and clinical psychology.

[5]  O. J. Dunn,et al.  The Treatment of Missing Values in Discriminant Analysis—I. The Sampling Experiment , 1972 .

[6]  E. Beale,et al.  Missing Values in Multivariate Analysis , 1975 .

[7]  Howard Wainer,et al.  Estimating Coefficients in Linear Models: It Don't Make No Nevermind , 1976 .

[8]  Jae-On Kim,et al.  The Treatment of Missing Data in Multivariate Analysis , 1977 .

[9]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[10]  R. Dawes.  Judgment under uncertainty: The robust beauty of improper linear models in decision making , 1979 .

[11]  Randall S. Schuler,et al.  Role conflict and ambiguity scales: Reality or artifacts? , 1983 .

[12]  Ingram Olkin,et al.  Incomplete data in sample surveys. Vol. 1: report and case studies , 1983 .

[13]  Ingram Olkin,et al.  Incomplete data in sample surveys , 1985 .

[14]  M. Raymond.  Missing Data in Evaluation Research , 1986 .

[15]  Naresh K. Malhotra,et al.  Analyzing Marketing Research Data with Incomplete Information on the Dependent Variable , 1987 .

[16]  Paul E. Spector,et al.  Unemployment, job satisfaction, and employee turnover: A meta-analytic test of the Muchinsky model. , 1987 .

[17]  Mark R. Raymond,et al.  A Comparison of Methods for Treating Incomplete Data in Selection Research , 1987 .

[18]  N M Laird,et al.  Missing data in longitudinal studies. , 1988, Statistics in medicine.

[19]  R. Little.  Missing-Data Adjustments in Large Surveys , 1988 .

[20]  I G Sande,et al.  Missing-Data Adjustments in Large Surveys: Comment , 1988 .

[21]  Steven D. Jones,et al.  Effects of group feedback, goal setting, and incentives on organizational productivity. , 1988 .

[22]  Roderick J. A. Little,et al.  The Analysis of Social Science Data with Missing Values , 1989 .

[23]  D. Rubin,et al.  Multiple Imputation for Nonresponse in Surveys , 1989 .

[24]  R. Gene Hoffman,et al.  LARGE‐SCALE PREDICTOR VALIDATION IN PROJECT A: DATA COLLECTION PROCEDURES AND DATA BASE PREPARATION , 1990 .

[25]  C H Brown,et al.  Protecting against nonrandomly missing data in longitudinal studies. , 1990, Biometrics.

[26]  L. J. Williams,et al.  Job Satisfaction and Organizational Commitment as Predictors of Organizational Citizenship and In-Role Behaviors , 1991 .

[27]  Paul A. Ruud,et al.  Extensions of estimation methods using the EM algorithm , 1991 .

[28]  Murray R. Barrick,et al.  THE BIG FIVE PERSONALITY DIMENSIONS AND JOB PERFORMANCE: A META-ANALYSIS , 1991 .

[29]  John P. Meyer,et al.  Development of Organizational Commitment During the First Year of Employment: A Longitudinal Study of Pre- and Post-Entry Influences , 1991 .

[30]  Lois E. Tetrick,et al.  A Construct Validity Study of the Survey of Perceived Organizational Support , 1991 .

[31]  J. Mathieu.  A cross-level nonrecursive model of the antecedents of organizational commitment and satisfaction. , 1991 .

[32]  J. Graham,et al.  Evaluating interventions with differential attrition: the importance of nonresponse mechanisms and use of follow-up data. , 1993, The Journal of applied psychology.

[33]  P. Roth.  MISSING DATA: A CONCEPTUAL REVIEW FOR APPLIED PSYCHOLOGISTS , 1994 .

[34]  Fred S. Switzer,et al.  A Monte Carlo Analysis of Missing Data Techniques in a HRM Setting , 1995 .

[35]  Timothy R. Hinkin.  A Review of Scale Development Practices in the Study of Organizations , 1995 .

[36]  Philip L. Roth,et al.  The impact of four missing data techniques on validity estimates in human resource management , 1996 .

[37]  Ken Baker,et al.  Data Fusion: An Appraisal and Experimental Evaluation , 1997 .

[38]  M. Audrey Korsgaard,et al.  BEYOND HELPING: DO OTHER-ORIENTED VALUES HAVE BROADER IMPLICATIONS IN ORGANIZATIONS? , 1997 .

[39]  R. Downey,et al.  Missing data in Likert ratings: A comparison of replacement methods. , 1998, The Journal of general psychology.

[40]  Deborah M. Switzer,et al.  Systematic Data Loss in HRM Settings: A Monte Carlo Analysis , 1998 .

[41]  P. Roth,et al.  The effect of Method Variance on Relationships Between the Work Ethic and Individual Difference Variables , 1999 .