Assessing spurious "moderator effects": Illustrated substantively with the hypothesized ("synergistic") relation between spatial and mathematical ability.

The traditional methodology for assessing moderator variables (hierarchical multiple regression analysis) is examined. Possible drawbacks of this technique for corroborating psychological theories (cf. Busemeyer & Jones, 1983), are illustrated empirically on the basis of an analysis of 400,000 subjects. This article tested a well-known (and currently popular) substantive hypothesis: A synergistic relation exists between mathematical ability and spatial visualization in the prediction and development of sophisticated levels of advanced mathematics. Using the traditional methodology, this hypothesis was confirmed; however, on further analysis, using a more systematic approach, it was demonstrated that this finding was spurious. Suggestions are offered for modifying the traditional methodology used for assessing moderator effects (for both applied and theoretical purposes). These amount to ways for minimizing Type I and Type II errors.

[1]  M. W. Richardson,et al.  The theory of the estimation of test reliability , 1937 .

[2]  D. R. Saunders.  Moderator Variables in Prediction , 1956 .

[3]  E. Ghiselli.  Differentiation of individuals in terms of their predictability. , 1956 .

[4]  J. Fisher.  The twisted pear and the prediction of behavior. , 1959 .

[5]  Edwin E. Ghiselli,et al.  The Prediction of Predictability , 1960 .

[6]  Norman Frederiksen,et al.  Replication of a Study of Differential Predictability , 1960 .

[7]  R. Berdie,et al.  Intra-Individual Variability and Predictability , 1961 .

[8]  J. C. Flanagan.  Design for a Study of American Youth. , 1962 .

[9]  Peter Secretan,et al.  Stanford Binet Intelligence Scale: Manual for the Third Revision Form L-M , 1963, Mental Health.

[10]  Edwin E. Ghiselli,et al.  Moderating effects and differential reliability and validity. , 1963 .

[11]  J. Sherman,et al.  Problem of sex differences in space perception and aspects of intellectual functioning. , 1967, Psychological review.

[12]  P. Meehl.  Theory-Testing in Psychology and Physics: A Methodological Paradox , 1967, Philosophy of Science.

[13]  D. Lykken.  Statistical significance in psychological research. , 1968, Psychological bulletin.

[14]  Jacob Cohen.  Multiple regression as a general data-analytic system. , 1968 .

[15]  S. Zedeck.  Problems with the use of "moderator" variables. , 1971 .

[16]  J. S. Wiggins,et al.  Personality and Prediction: Principles of Personality Assessment , 1973 .

[17]  David Wechsler,et al.  Manual for the Wechsler intelligence scale for children , 1974 .

[18]  F. Schmidt,et al.  Racial and ethnic bias in psychological tests: Divergent implications of two definitions of test bias. , 1974 .

[19]  F. Schmidt,et al.  Critical analysis of the statistical and ethical implications of various definitions of test bias. , 1976 .

[20]  John E. Hunter,et al.  Development of a general solution to the problem of validity generalization. , 1977 .

[21]  John E. Hunter,et al.  Differential and single-group validity of employment tests by race: A critical analysis of three recent studies. , 1978 .

[22]  George W. Bohrnstedt,et al.  The Reliability of Products of Two Random Variables , 1978 .

[23]  A. Tellegen,et al.  The relationship between androgyny and subjective indicators of emotional well-being. , 1981 .

[24]  A. Tellegen,et al.  Recognizing individual differences in predictive structure , 1982 .

[25]  H. J. Arnold.  Moderator variables: A clarification of conceptual, analytic, and psychometric issues , 1982 .

[26]  L. E. Jones,et al.  Analysis of multiplicative combination rules when the causal variables are measured with error. , 1983 .

[27]  (The Androgyny Dimension: A Comment on Stokes, Childs, and Fuehrer: And a Response.). , 1983 .

[28]  James N. Butcher,et al.  Masculinity, femininity, and androgyny viewed and assessed as distinct concepts. , 1983 .

[29]  A. Tellegen,et al.  Some methodological comments on labels, traits, interaction, and types in the study of "femininity" and "masculinity": Reply to Spence. , 1983 .

[30]  L. R. Goldberg,et al.  A failure to replicate the Bem and Allen study of individual differences in cross-situational consistency. , 1984, Journal of personality and social psychology.

[31]  R. Dawis,et al.  A psychological theory of work adjustment : an individual-differences model and its applications , 1984 .

[32]  H. J. Arnold,et al.  Testing moderator variable hypotheses: A reply to stone and hollenbeck , 1984 .

[33]  D. Jackson,et al.  Idiographic measurement strategies for personality and prediction: Some unredeemed promissory notes , 1985 .

[34]  L. Cronbach.  Statistical tests for moderator variables: flaws in analyses recently proposed , 1987 .

[35]  S. Paunonen.  Trait Relevance and the Differential Predictability of Behavior , 1988 .

[36]  F. Schmidt.  The Problem of Group Differences in Ability Test Scores in Employment Selection , 1988 .

[37]  Auke Tellegen,et al.  The Analysis of Consistency in Personality Assessment , 1988 .

[38]  C. Benbow.  Some of the pathological assumptions in the sciences of gender , 1988, Behavioral and Brain Sciences.

[39]  L. Humphreys,et al.  A broadly based analysis of mathematical giftedness , 1990 .