Finding and Correcting Flawed Research Literatures

Humanistic psychology has always viewed scientific psychology with skepticism. Good reasons for this skepticism continuously appear. One is then left with the choice, "Is a scientific approach to humans inherently wrongheaded?" or "Is scientific psychology an imperfect but improving enterprise?" This article reviews another domain where research in scientific psychology proves misleading. Suppose a psychologist was asked a question such as, "Is psychotherapy effective?" or "Is remote intercessory prayer effective?" or "Do humans possess psychic powers?" How might a psychologist reply? The most common strategy would be to conduct a meta-analysis over the relevant research literature and report the results. In all 3 cases (i.e., psychotherapy, efficacy of remote intercessory prayer, and telepathic powers) the answer would be a significant, positive effect size, suggesting that all 3 are real, efficacious phenomena. Unfortunately, in at least 2 of the 3 cases, the literature likely gives an incorrect answer ...

[1]  W. N. Schoenfeld,et al.  Principles of Psychology , 2007 .

[2]  L. Storm,et al.  Does psi exist? Comments on Milton and Wiseman's (1999) meta-analysis of ganzfeld research. , 2001, Psychological bulletin.

[3]  S D Walter,et al.  A comparison of methods to detect publication bias in meta‐analysis , 2001, Statistics in medicine.

[4]  S. Maxwell,et al.  The proof of the pudding: an illustration of the relative strengths of null hypothesis, meta-analysis, and Bayesian analysis. , 2000, Psychological methods.

[5]  R. Tweedie,et al.  A Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis , 2000 .

[6]  Lois Hetland,et al.  Listening to Music Enhances Spatial-Temporal Reasoning: Evidence for the "Mozart Effect.". , 2000 .

[7]  J. O’Keefe,et al.  A randomized, controlled trial of the effects of remote, intercessory prayer on outcomes in patients admitted to the coronary care unit. , 1999, Archives of internal medicine.

[8]  P. Gollwitzer.  Implementation intentions: Strong effects of simple plans. , 1999 .

[9]  Helena C. Kraemer,et al.  Advantages of excluding underpowered studies in meta-analysis: Inclusionist versus exclusionist viewpoints. , 1998 .

[10]  Larry V. Hedges,et al.  Estimating Effect Size Under Publication Bias: Small Sample Properties and Robustness of a Random Effects Selection Model , 1996 .

[11]  F. Wertz.  Representations of the “third force” in history of psychology textbooks , 1992 .

[12]  Amedeo Giorgi.  The idea of human science , 1992 .

[13]  R. Decarvalho.  The institutionalization of humanistic psychology , 1992 .

[14]  M. Mair.  Between Psychology and Psychotherapy: A Poetics of Experience , 1989 .

[15]  R C Byrd,et al.  Positive Therapeutic Effects of Intercessory Prayer in a Coronary Care Unit Population , 1988, Southern medical journal.

[16]  Joel B. Greenhouse,et al.  Selection Models and the File Drawer Problem , 1988 .

[17]  G. Howard.  Improving Methodology via Research on Research Methods. , 1982 .

[18]  Richard L. Wiener,et al.  Is a Behavioral Measure the Best Estimate of Behavioral Parameters? Perhaps Not , 1980 .

[19]  R. Rosenthal.  The file drawer problem and tolerance for null results , 1979 .

[20]  Scott E. Maxwell,et al.  Internal Invalidity in Pretest-Posttest Self-Report Evaluations and a Re-evaluation of Retrospective Pretests , 1979 .

[21]  P. Meehl.  Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. , 1978 .

[22]  Richard L. Tweedie,et al.  A Non-parametric "Trim and Fill" Method of Assessing Publication Bias in Meta-analysis , 1998 .

[23]  D. Bem,et al.  Does psi exist? Replicable evidence for an anomalous process of information transfer. , 1994 .

[24]  J. Werry,et al.  The effectiveness of psychotherapy. , 1993, Journal of the American Academy of Child and Adolescent Psychiatry.

[25]  G. Howard,et al.  Can there be an empirical science of volitional action , 1986 .

[26]  Christopher M. Aanstoos.  Psychology as a human science. , 1985 .

[27]  Gene V. Glass,et al.  The benefits of psychotherapy , 1980 .

[28]  David W. Bates,et al.  Information Transfer , 2021, Industry, Innovation and Infrastructure.