The Relative Validity of Inferences About Mediation as a Function of Research Design Characteristics

Tests of assumed mediation models are common in the organizational sciences. However, the validity of inferences about mediation is a function of experimental design and the setting of a study. Regrettably, most tests of mediation have relied on the application of so-called ``causal modeling'' techniques to data from nonexperimental studies. As we demonstrate, inferences about the validity of assumed mediation models are highly suspect when they are based on the findings of nonexperimental research. One of the many reasons for this is the failure of the model being tested to be consistent with reality. Valid research-based inferences about mediation are possible. However, inferences from such tests are most likely to be valid when they are based on research that uses randomized experimental designs. Strategies for conducting research using these and other designs are described. Finally, we offer a set of conclusions and recommendations that stem from our analysis.

[1]  C. A. Higgins,et al.  The effect of applicant influence tactics on recruiter perceptions of fit and hiring recommendations: a field study. , 2004, The Journal of applied psychology.

[2]  T. Cook,et al.  Quasi-experimentation: Design & analysis issues for field settings , 1979 .

[3]  C. Kovner,et al.  Non-experimental designs , 1986 .

[4]  P. Holland.  Statistics and Causal Inference , 1985 .

[5]  Richard,et al.  Motivation through the Design of Work: Test of a Theory. , 1976 .

[6]  H. Blalock.  Causal Inferences in Nonexperimental Research , 1966 .

[7]  Ian R. Gellatly,et al.  Perceived performance norm as a mediator in the effect of assigned goal on personal goal and task performance. , 1988 .

[8]  R. Montague,et al.  Logic : Techniques of Formal Reasoning , 1964 .

[9]  Steven P. Brown,et al.  The attenuating effect of role overload on relationships linking self-efficacy and goal level to work performance. , 2005, The Journal of applied psychology.

[10]  D. A. Kenny,et al.  Correlation and Causation. , 1982 .

[11]  Merrill Carlsmith.  Methods of research in social psychology , 1976 .

[12]  L. Cronbach,et al.  Construct validity in psychological tests. , 1955, Psychological bulletin.

[13]  A. Goldberger,et al.  Causal Models in the Social Sciences. , 1972 .

[14]  Murray R. Barrick,et al.  Team Structure and Performance: Assessing the Mediating Role of Intrateam Process and the Moderating Role of Task Type , 2000 .

[15]  A. Baron.  Experimental Designs , 1990, The Behavior analyst.

[16]  E. Kelloway.  Using LISREL for Structural Equation Modeling: A Researcher′s Guide , 1998 .

[17]  W. Shadish,et al.  Experimental and Quasi-Experimental Designs for Generalized Causal Inference , 2001 .

[18]  David Rogosa,et al.  Casual Models Do Not Support Scientific Conclusions: A Comment in Support of Freedman , 1987 .

[19]  Eugene F. Stone‐Romero,et al.  The Relative Validity and Usefulness of Various Empirical Research Designs , 2008 .

[20]  William G. Cochran,et al.  Experimental Designs, 2nd Edition , 1950 .

[21]  L. Delbeke.  Quasi-experimentation - design and analysis issues for field settings - cook,td, campbell,dt , 1980 .

[22]  Jason Millman,et al.  Research on human behavior;: A systematic guide to method , 1972 .

[23]  Comments on Professor Freedman’s Paper , 1987 .

[24]  D. A. Kenny,et al.  Correlation and causality , 1979 .

[25]  V. Vroom.  Work and motivation , 1964 .

[26]  M. L. Blum,et al.  Industrial Psychology: Its Theoretical and Social Foundations , 1968 .

[27]  S. Mellor,et al.  Self-efficacy as a mediator of the relationship between perceived union barriers and women's participation in union activities. , 1997, The Journal of applied psychology.

[28]  M. Heilman,et al.  Same behavior, different consequences: reactions to men's and women's altruistic citizenship behavior. , 2005, The Journal of applied psychology.

[29]  L. Berkowitz,et al.  External validity is more than skin deep: Some answers to criticisms of laboratory experiments. , 1982 .

[30]  Edwin A. Locke,et al.  Generalizing from Laboratory to Field Settings: Research Findings from Industrial-Organizational Psychology, Organizational Behavior, and Human Resource Management. , 1987 .

[31]  G. Freytag.  [CORRELATION AND CAUSALITY]. , 1964, Psychiatrie, Neurologie, und medizinische Psychologie.

[32]  I. Ajzen,et al.  Understanding Attitudes and Predicting Social Behavior , 1980 .

[33]  D. A. Kenny,et al.  The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. , 1986, Journal of personality and social psychology.

[34]  Michael T. Brannick,et al.  Critical comments on applying covariance structure modeling , 1995 .

[35]  I. Ajzen,et al.  Belief, Attitude, Intention, and Behavior: An Introduction to Theory and Research , 1977 .

[36]  L. James,et al.  Mediators, Moderators, and Tests for Mediation. , 1984 .

[37]  Roger E. Millsap,et al.  Structural equation modeling: A user's guide. , 2002 .

[38]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[39]  K. Glenk,et al.  Validity and Reliability , 2008, Environmental Valuation with Discrete Choice Experiments.

[40]  G A Morgan,et al.  Quasi-experimental designs. , 2000, Journal of the American Academy of Child and Adolescent Psychiatry.

[41]  A. Rapoport.  Review of Correlation and Causality. , 1980 .

[42]  Kenneth A. Bollen,et al.  Structural Equations with Latent Variables , 1989 .

[43]  Stephen Gough,et al.  Non-Experimental Designs , 2004 .

[44]  Lyman W. Porter,et al.  Managerial attitudes and performance , 1968 .

[45]  E. Pedhazur.  Multiple Regression in Behavioral Research: Explanation and Prediction , 1982 .

[46]  Patrick J. Rosopa,et al.  INFERENCE PROBLEMS WITH HIERARCHICAL MULTIPLE REGRESSION-BASED TESTS OF MEDIATING EFFECTS , 2004 .

[47]  Eugene F. Stone.  Research methods in organizational behavior , 1978 .

[48]  John E. Mathieu,et al.  Clarifying conditions and decision points for mediational type inferences in Organizational Behavior , 2006 .

[49]  D. Freedman.  As Others See Us: A Case Study in Path Analysis , 1987 .

[50]  Chun Hui,et al.  Psychological contract and organizational citizenship behavior in China: investigating generalizability and instrumentality. , 2004, The Journal of applied psychology.