Some Methodological and Statistical “Bugs” in Research on Children’s Learning

This chapter provides me with the opportunity to discuss a number of methodological and statistical “bugs” that I have detected creeping into psychological research in general, and into research on children’s learning in particular. Naturally, one cannot hope to exterminate all such bugs with but a single essay. Rather, it is hoped that this chapter will leave a trail of pellets that is sufficiently odorific to get to the source of these potentially destructive little creatures. It also goes without saying that different people in this trade have different entomological lists that they would like to see presented. Although all cannot be presented here, I intend to introduce you to nearly 20 of my own personal favorites. At the same time, it must be stated at the outset that present space limitations do not permit a complete specification and resolution of the problems that these omnipresent bugs can create for cognitive-developmental researchers. Consequently, in most cases I will only allude to a problem and its potential remedies, placing the motivation for additional inquiry squarely in the lap of the curious reader.

[1]  Dusan Hadzi-Pavlovic,et al.  Simultaneous test procedures and the choice of a test statistic in MANOVA. , 1983 .

[2]  B. J. Winer.  Statistical Principles in Experimental Design , 1992 .

[3]  Mark Appelbaum,et al.  Bias in the Analysis of Repeated-Measures Designs: Some Alternative Approaches. , 1973 .

[4]  Bradley E. Huitema,et al.  The analysis of covariance and alternatives , 1980 .

[5]  G. Glass,et al.  The Experimental Unit in Statistical Analysis , 1969 .

[6]  Selby H Evans,et al.  Misuse of analysis of covariance when treatment effect and covariate are confounded. , 1968, Psychological bulletin.

[7]  Scott E. Maxwell,et al.  Another look at ANCOVA versus blocking. , 1984 .

[8]  W. D. Rohwer.  Elaboration and learning in childhood and adolescence. , 1973, Advances in child development and behavior.

[9]  L. A. Marascuilo,et al.  Appropriate Post Hoc Comparisons for Interaction and Nested Hypotheses in Analysis of Variance Designs: The Elimination of Type IV Errors1 , 1970 .

[10]  A. Lubin.  The Interpretation of Significant Interaction , 1961 .

[11]  G. Loftus.  On interpretation of interactions , 1978 .

[12]  Joel R. Levin,et al.  Multivariate statistics in the social sciences : a researcher's guide , 1985 .

[13]  John Seely Brown,et al.  Diagnostic Models for Procedural Bugs in Basic Mathematical Skills , 1978, Cogn. Sci..

[14]  Chester L. Olson,et al.  Comparative Robustness of Six Tests in Multivariate Analysis of Variance , 1974 .

[15]  F. Lord.  A paradox in the interpretation of group comparisons. , 1967, Psychological bulletin.

[16]  R. Bogartz,et al.  On the Meaning of Statistical Interactions. , 1976 .

[17]  R. Boruch,et al.  3 – Making the Case for Randomized Assignment to Treatments by Considering the Alternatives: Six Ways in Which Quasi-Experimental Evaluations In Compensatory Education Tend to Underestimate Effects , 1975 .

[18]  D. Campbell,et al.  EXPERIMENTAL AND QUASI-EXPERIMENT Al DESIGNS FOR RESEARCH , 2012 .

[19]  Joel R. Levin,et al.  The Simultaneous Investigation of Interaction and Nested Hypotheses in Two-Factor Analysis of Variance Designs , 1976 .

[20]  Joel R. Levin,et al.  3 – N = Nonparametric Randomization Tests , 1978 .

[21]  Jorge L. Mendoza,et al.  Analysis of repeated measurements. , 1979 .

[22]  Joel R. Levin,et al.  Assessing the classroom potential of the keyword method. , 1979 .

[23]  J. Levin,et al.  Classroom Aptitude-by-Treatment Interactions: An Alternative Analysis Strategy , 1984 .

[24]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[25]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[26]  Joel R. Levin,et al.  DETERMINING SAMPLE SIZE FOR PLANNED AND POST HOC ANALYSIS OF VARIANCE COMPARISONS1 , 1975 .

[27]  Robert S. Barcikowski,et al.  Statistical Power with Group Mean as the Unit of Analysis , 1981 .

[28]  A. Greenwald.  Consequences of Prejudice Against the Null Hypothesis , 1975 .

[29]  T. Salthouse,et al.  Memory Methodology in Maturity , 1985 .

[30]  Ronald C. Serlin,et al.  QUANTITATIVE METHODS IN PSYCHOLOGY Comparison of ANOVA Alternatives Under Variance Heterogeneity and Specific Noncentrality Structures , 1986 .

[31]  Janet D. Elashoff,et al.  Analysis of Covariance: A Delicate Instrument1 , 1969 .

[32]  Paul A. Games,et al.  Nesting, Crossing, Type IV Errors, and the Role of Statistical Models , 1978 .

[33]  G. Glass.  9: Integrating Findings: The Meta-Analysis of Research , 1977 .

[34]  P. Baltes,et al.  Life-span Developmental Psychology: Introduction To Research Methods , 1977 .

[35]  L. A. Marascuilo,et al.  Nonparametric and Distribution-Free Methods for the Social Sciences , 1977 .

[36]  Denton E. Morrison,et al.  The Significance Test Controversy , 1972 .

[37]  L. Hubert,et al.  Hypothesis-Testing Procedures in Repeated-Measures Designs: On the Road Map Not Taken. , 1977 .

[38]  J. Overall,et al.  Concerning least squares analysis of experimental data. , 1969 .

[39]  James Jaccard,et al.  Pairwise multiple comparison procedures: A review. , 1984 .

[40]  K. Gabriel,et al.  SIMULTANEOUS TEST PROCEDURES-SOME THEORY OF MULTIPLE COMPARISONS' , 1969 .

[41]  L. Humphreys.  The Statistics of Failure to Replicate: A Comment on Buriel's (1978) Conclusions. , 1980 .

[42]  Curtis Hardyck,et al.  Error rates for multiple comparison methods: Some evidence concerning the frequency of erroneous conclusions. , 1969 .

[43]  G. Glass,et al.  Consequences of Failure to Meet Assumptions Underlying the Fixed Effects Analyses of Variance and Covariance , 1972 .

[44]  J. Levin,et al.  Coherent Analysis-of-Variance Hypothesis-Testing Strategies: A General Model , 1982 .

[45]  Mark L. Howe,et al.  Developmental Invariance in a Mathematical Model of Associative Learning. , 1980 .

[46]  Michael Pressley,et al.  Understanding Mnemonic Imagery Effects: A Dozen "Obvious" Outcomes. , 1981 .

[47]  J. Gaito,et al.  UNEQUAL INTERVALS AND UNEQUAL N IN TREND ANALYSES. , 1965, Psychological bulletin.