Meta-Analysis of Pragmatic and Theoretical Research: A Critique

Meta-analysis refers to a set of statistical procedures used to summarize and integrate many empirical studies that focus on one issue. This numerical method of integrating research findings is said to be superior to the narrative type of reviews because it is more objective, reliable, and rigorous. Moreover, the meta-analytic approach is supposedly capable of resolving research controversies, strengthening empirical hypotheses, and discovering new relationships among variables. In this study, these claims are examined and found to be wanting. Some objections to the use of meta-analysis as a means of substantiating theoretical assertions are raised with reference to the rationale of experimentation and to how knowledge evolves. It is concluded that it is inappropriate to apply meta-analysis to integrate theoretical research.

[1]  R. Harré,et al.  The explanation of social behaviour , 1973 .

[2]  T. Cook,et al.  What differentiates meta-analysis from other forms of review? , 1981 .

[3]  D. Kahneman.  Method, findings, and theory in studies of visual masking. , 1968, Psychological bulletin.

[4]  J. Mintz,et al.  Integrating research evidence: a commentary on meta-analysis. , 1983, Journal of consulting and clinical psychology.

[5]  D. Rubin,et al.  Comparing Effect Sizes of Independent Studies , 1982 .

[6]  Harris Cooper,et al.  Statistically Combining Independent Studies: A Meta-Analysis of Sex Differences in Conformity Research , 1979 .

[7]  L. Berkowitz,et al.  External validity is more than skin deep: Some answers to criticisms of laboratory experiments. , 1982 .

[8]  Critique of Cooper's meta-analytic assessment of the findings on sex differences in conformity behavior. , 1980 .

[9]  Kenneth A. Feldman,et al.  Using the Work of Others: Some Observations on Reviewing and Integrating , 1971 .

[10]  D. Shapiro,et al.  Comparative therapy outcome research: methodological implications of meta-analysis. , 1983, Journal of consulting and clinical psychology.

[11]  Laura C. Leviton,et al.  Reviewing the literature: A comparison of traditional methods with meta-analysis. , 1980 .

[12]  M. Coltheart,et al.  Iconic memory and visible persistence , 1980, Perception & psychophysics.

[13]  P. Gallo.  Meta-analysis: A mixed meta-phor? , 1978 .

[14]  Donald B. Rubin,et al.  Comparing Significance Levels of Independent Studies , 1979 .

[15]  Siu L. Chow,et al.  Science, Ecological Validity and Experimentation , 1987 .

[16]  D. P. Hartmann,et al.  A critical appraisal of meta-analysis. , 1982, The British journal of clinical psychology.

[17]  G. Glass.  Primary, Secondary, and Meta-Analysis of Research1 , 1976 .

[18]  P. Meehl.  Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. , 1978 .

[19]  R. Rosenthal,et al.  Assessing the statistical and social importance of the effects of psychotherapy. , 1983, Journal of consulting and clinical psychology.

[20]  G. Glass,et al.  Meta-analysis in social research , 1981 .

[21]  S. Rachman,et al.  Meta-analysis and the evaluation of psychotherapy outcome: limitations and liabilities. , 1983, Journal of consulting and clinical psychology.

[22]  D. Mook,et al.  In defense of external invalidity. , 1983 .

[23]  P. Cozby,et al.  Methods in behavioral research , 1977 .

[24]  D. Campbell,et al.  EXPERIMENTAL AND QUASI-EXPERIMENT Al DESIGNS FOR RESEARCH , 2012 .

[25]  Michael J. Strube,et al.  Meta-Analysis and Cross-Cultural Comparison , 1981 .

[26]  R. Rosenthal,et al.  Statistical versus traditional procedures for summarizing research findings. , 1980, Psychological bulletin.

[27]  R. Haber.  The impending demise of the icon: A critique of the concept of iconic storage in visual information processing , 1983, Behavioral and Brain Sciences.

[28]  Experimental Psychology: Rationale, Procedures and Issues , 1987 .

[29]  D. P. Hartmann,et al.  Meta-analysis: Techniques, applications, and functions. , 1983 .

[30]  Susan Presby,et al.  Overly broad categories obscure important differences between therapies. , 1978 .

[31]  M. L. Smith,et al.  Meta-analysis of psychotherapy outcome studies. , 1977, The American psychologist.

[32]  G. Glass,et al.  An apology for research integration in the study of psychotherapy. , 1983, Journal of consulting and clinical psychology.

[33]  S. Rachman,et al.  The effects of psychological therapy , 1980 .

[34]  P. F. Secord,et al.  Implications for psychology of the new philosophy of science. , 1983 .

[35]  Donald B. Rubin,et al.  Interpersonal expectancy effects: the first 345 studies , 1978, Behavioral and Brain Sciences.

[36]  Robert Rosenthal,et al.  The Pygmalion Effect Lives. , 1973 .

[37]  Donald B. Rubin,et al.  A Simple, General Purpose Display of Magnitude of Experimental Effect , 1982 .

[38]  Gene V. Glass,et al.  The benefits of psychotherapy , 1980 .

[39]  H. Eysenck.  An exercise in mega-silliness. , 1978 .

[40]  Harris Cooper,et al.  Scientific Guidelines for Conducting Integrative Research Reviews , 1982 .

[41]  Gregg B. Jackson,et al.  Methods for Integrative Reviews , 1980 .

[42]  R. Rosenthal.  Combining results of independent studies. , 1978 .

[43]  M Coltheart,et al.  Iconic memory: A reply to Professor Holding , 1975, Memory & cognition.

[44]  R. Light,et al.  Accumulating Evidence: Procedures for Resolving Contradictions among Different Research Studies. , 1971 .