Data analysis: continuing issues in the everyday analysis of psychological data.

CONTENTS INTRODUCTION . . . . . . . . . . . . . . . . . ... . . . . . . . . . . . . . . . . . . . .. . .. . . . . . . . . . .. . . . . .... . . . . . . . . . . . . . . . . ... ..... .... ... . . . . . . . . . . . . . 433 THE MODEL COMPARISON APPROACH TO DATA ANALYSIS . . . . . ...... . . . . . . . . .... .... ...... . . 434 Significance Tests and Effect Estimates.. . . . . . . . . .. . . ... . .. . . .. . . . . . . . . . . . . . . . . .. . .. . . . . . . . . . . . . . . ... . . . . . . . . . . . . . . . . . 437 Focused Comparisons.. . . . . . .. .. . . . . .... . ... . . . . . . . . . . . . . . . . . . . . .. ........ 440 Statistical Power . . . . . . . . . . . . . . . . . . . . . .. .. . . . . . . . . . .... . . ...... .... . ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . .... .... .... 445 Interactions 446 ASSUMPTIONS: VIOLATIONS AND SOLUTIONS . .. . . .... ... . .... . . . . . . . . . . . .. . . .... . . . . . . . . . . . . . 452 Distributional Problems... . . . . . . .. . . . . . .. . . . . . . . . . . . . . .... .... .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452 Remedies/or Distributional Problems.. . . . . . . . . . . .. . . . . . . . . . . . . . . ...... . . . . . . . .... ...... . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . 455 Nonindepen dence 0/ Observations.. . . . ... . . . . . . . . . . . . . . . ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . ... 459 CONCLUSIONS.... .... ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . .... .... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 460

[1]  L. S. Feldt.  A comparison of the precision of three experimental designs employing a concomitant variable , 1958 .

[2]  William S. Cleveland,et al.  Visualizing Data , 1993 .

[3]  Anthony S. Bryk,et al.  Application of Hierarchical Linear Models to Assessing Change , 1987 .

[4]  Rand R. Wilcox,et al.  New designs in analysis of variance. , 1987 .

[5]  Jan de Leeuw,et al.  Random Coefficient Models for Multilevel Analysis , 1986 .

[6]  R. Tibshirani,et al.  Generalized additive models for medical research , 1986, Statistical methods in medical research.

[7]  D. A. Kenny,et al.  Consequences of violating the independence assumption in analysis of variance. , 1986 .

[8]  R. Friedrich.  In Defense of Multiplicative Terms In Multiple Regression Equations , 1982 .

[9]  Geoffrey R. Loftus,et al.  On the Tyranny of Hypothesis Testing in the Social Sciences , 1991 .

[10]  Deborah A. Prentice,et al.  When small effects are impressive , 1992 .

[11]  L. Humphreys,et al.  Assessing spurious "moderator effects": Illustrated substantively with the hypothesized ("synergistic") relation between spatial and mathematical ability. , 1990, Psychological bulletin.

[12]  D. Rubin,et al.  Statistical Analysis: Summarizing Evidence Versus Establishing Facts , 1985 .

[13]  A. Tamhane,et al.  Multiple Comparison Procedures , 1989 .

[14]  L. Cronbach.  Statistical tests for moderator variables: flaws in analyses recently proposed , 1987 .

[15]  H. Huynh,et al.  Conditions under Which Mean Square Ratios in Repeated Measurements Designs Have Exact F-Distributions , 1970 .

[16]  Trevor Hastie,et al.  Statistical Models in S , 1991 .

[17]  H. Scheffé,et al.  The Analysis of Variance , 1960 .

[18]  D. A. Kenny,et al.  Estimating the effects of social interventions , 1981 .

[19]  Jacob Cohen,et al.  The statistical power of abnormal-social psychological research: a review. , 1962, Journal of abnormal and social psychology.

[20]  R. Dawes.  Monotone interactions: It's even simpler than that , 1990, Behavioral and Brain Sciences.

[21]  Jacob Cohen.  Multiple regression as a general data-analytic system. , 1968 .

[22]  G. Gigerenzer,et al.  Do studies of statistical power have an effect on the power of studies , 1989 .

[23]  David Rogosa,et al.  A growth curve approach to the measurement of change. , 1982 .

[24]  O. Pendleton.  Influential observations in the analysis of variance , 1985 .

[25]  Michael R. Harwell,et al.  Summarizing Monte Carlo Results in Methodological Research: The One- and Two-Factor Fixed Effects ANOVA Cases , 1992 .

[26]  B Efron,et al.  Statistical Data Analysis in the Computer Age , 1991, Science.

[27]  L. E. Jones,et al.  Analysis of multiplicative combination rules when the causal variables are measured with error. , 1983 .

[28]  Stephen A. Shumake,et al.  Animal Welfare and the Statistical Consultant , 1993 .

[29]  Robert Rosenthal,et al.  How are we doing in soft psychology , 1990 .

[30]  R. Folger.  Significance Tests and the Duplicity of Binary Decisions , 1989 .

[31]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[32]  P. McCullagh,et al.  Generalized Linear Models , 1984 .

[33]  D. Newman,et al.  THE DISTRIBUTION OF RANGE IN SAMPLES FROM A NORMAL POPULATION, EXPRESSED IN TERMS OF AN INDEPENDENT ESTIMATE OF STANDARD DEVIATION , 1939 .

[34]  D. Bakan,et al.  The test of significance in psychological research. , 1966, Psychological bulletin.

[35]  G. Glass,et al.  Consequences of Failure to Meet Assumptions Underlying the Fixed Effects Analyses of Variance and Covariance , 1972 .

[36]  R. Rosenthal,et al.  Contrast Analysis: Focused Comparisons in the Analysis of Variance , 1985 .

[37]  H. Huynh,et al.  Estimation of the Box Correction for Degrees of Freedom from Sample Data in Randomized Block and Split-Plot Designs , 1976 .

[38]  James V. Bradley,et al.  The insidious L-shaped distribution. , 1982 .

[39]  J. Tukey.  The Philosophy of Multiple Comparisons , 1991 .

[40]  W. W. Muir,et al.  Regression Diagnostics: Identifying Influential Data and Sources of Collinearity , 1980 .

[41]  D. A. Grant,et al.  Testing the null hypothesis and the strategy and tactics of investigating theoretical models. , 1962, Psychological review.

[42]  G. Box.  Some Theorems on Quadratic Forms Applied in the Study of Analysis of Variance Problems, II. Effects of Inequality of Variance and of Correlation Between Errors in the Two-Way Classification , 1954 .