Science or Art? How Aesthetic Standards Grease the Way Through the Publication Bottleneck but Undermine Science

The current crisis in psychological research involves issues of fraud, replication, publication bias, and false positive results. I argue that this crisis follows the failure of widely adopted solutions to psychology’s similar crisis of the 1970s. The untouched root cause is an information-economic one: Too many studies divided by too few publication outlets equals a bottleneck. Articles cannot pass through just by showing theoretical meaning and methodological rigor; their results must appear to support the hypothesis perfectly. Consequently, psychologists must master the art of presenting perfect-looking results just to survive in the profession. This favors aesthetic criteria of presentation in a way that harms science’s search for truth. Shallow standards of statistical perfection distort analyses and undermine the accuracy of cumulative data; narrative expectations encourage dishonesty about the relationship between results and hypotheses; criteria of novelty suppress replication attempts. Concerns about truth in research are emerging in other sciences and may eventually descend on our heads in the form of difficult and insensitive regulations. I suggest a more palatable solution: to open the bottleneck, putting structures in place to reward broader forms of information sharing beyond the exquisite art of present-day journal publication.

[1]  J. Wicherts,et al.  The Rules of the Game Called Psychological Science , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[2]  Lutz Bornmann,et al.  The Anna Karenina principle: A way of thinking about success in science , 2012, J. Assoc. Inf. Sci. Technol..

[3]  T. Sturm,et al.  Crisis discussions in psychology--New historical and philosophical perspectives. , 2012, Studies in history and philosophy of biological and biomedical sciences.

[4]  D. Sarewitz Beware the creeping cracks of bias , 2012, Nature.

[5]  Brian A. Nosek,et al.  Scientific Utopia: I. Opening Scientific Communication , 2012, ArXiv.

[6]  C. Begley,et al.  Drug development: Raise standards for preclinical cancer research , 2012, Nature.

[7]  Gregory Francis,et al.  Too good to be true: Publication bias in two prominent studies from experimental psychology , 2012, Psychonomic Bulletin & Review.

[8]  Henry L. Roediger,et al.  Psychology’s Woes and a Partial Cure: The Value of Replication , 2012 .

[9]  A. Ledgerwood,et al.  Short, Sweet, and Problematic? The Rise of the Short Report in Psychological Science , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[10]  Arina K Bones We Knew the Future All Along : Scientific Hypothesizing is Much More Accurate Than Other Forms of , 2012 .

[11]  A. Casadevall,et al.  Reforming Science: Methodological and Cultural Reforms , 2011, Infection and Immunity.

[12]  Etienne P LeBel,et al.  Fearing the Future of Empirical Psychology: Bem's (2011) Evidence of Psi as a Case Study of Deficiencies in Modal Research Practice , 2011, Review of General Psychology.

[13]  Leif D. Nelson,et al.  False-Positive Psychology , 2011, Psychological science.

[14]  F. Prinz,et al.  Believe it or not: how much can we rely on published data on potential drug targets? , 2011, Nature Reviews Drug Discovery.

[15]  E. Wagenmakers,et al.  Why psychologists must change the way they analyze their data: the case of psi: comment on Bem (2011). , 2011, Journal of personality and social psychology.

[16]  D. Bem Feeling the future: experimental evidence for anomalous retroactive influences on cognition and affect. , 2011, Journal of personality and social psychology.

[17]  E. H. Witte,et al.  Social Psychological Research: The Comparison of Four Journals , 2010 .

[18]  Celia M. Lombardi,et al.  Final Collapse of the Neyman-Pearson Decision Theoretic Framework and Rise of the neoFisherian , 2009 .

[19]  Celia M. Lombardi,et al.  Misprescription and misuse of one‐tailed tests , 2009 .

[20]  W. Sluckin,et al.  Novelty and Human Aesthetic Preferences , 2009 .

[21]  J. Ioannidis,et al.  Why Current Publication Practices May Distort Science , 2008, PLoS medicine.

[22]  G. Loewenstein,et al.  Exploring the Cold-to-Hot Empathy Gap in Smokers , 2008, Psychological science.

[23]  Adam D. Galinsky,et al.  PSYCHOLOGICAL SCIENCE Research Article The Merits of Unconscious Thought in Creativity , 2022 .

[24]  R. Rosenthal,et al.  Selective publication of antidepressant trials and its influence on apparent efficacy. , 2008, The New England journal of medicine.

[25]  J. Ioannidis,et al.  An exploratory test for an excess of significant findings , 2007, Clinical trials.

[26]  Jason Potts,et al.  What’s new in the economics of arts and culture? , 2007 .

[27]  D. Carpenter Institutional Review Boards, Regulatory Incentives, and Some Modest Proposals for Reform , 2006 .

[28]  M. Borenstein,et al.  Publication Bias in Meta-Analysis , 2006 .

[29]  G. Rhodes The evolutionary psychology of facial beauty. , 2006, Annual review of psychology.

[30]  J. Ioannidis Why Most Published Research Findings Are False , 2005, PLoS medicine.

[31]  Michael W. Kalichman,et al.  The Great Betrayal: Fraud in Science , 2005 .

[32]  N. Schwarz,et al.  Processing Fluency and Aesthetic Pleasure: Is Beauty in the Perceiver's Processing Experience? , 2004, Personality and social psychology review : an official journal of the Society for Personality and Social Psychology, Inc.

[33]  María Peñaranda-Ortega,et al.  The nature of social and personality psychology as reflected in JPSP, 1965-2000. , 2004, Journal of personality and social psychology.

[34]  F. Samelson What to do about fraud charges in science; or, will the Burt affair ever end? , 2004, Genetica.

[35]  G. Gigerenzer Mindless statistics , 2004 .

[36]  L. HARKing: Hypothesizing After the Results are Known , 2002 .

[37]  K C Klauer,et al.  Does Sunshine Prime Loyal? Affective Priming in the Naming Task , 2001, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[38]  J. Grice,et al.  The Evolution of Sex Differences in Jealousy: Failure to Replicate Previous Results , 2000 .

[39]  Leland Wilkinson,et al.  Statistical Methods in Psychology Journals Guidelines and Explanations , 2005 .

[40]  Amy M. Buddie,et al.  Twenty Years of PSPB: Trends in Content, Design, and Analysis , 1999 .

[41]  L. Hedges,et al.  The Handbook of Research Synthesis , 1995 .

[42]  N. J. Mackintosh,et al.  Cyril Burt: Fraud or framed? , 1995 .

[43]  William R. Shadish,et al.  The Social Psychology of Science , 1995 .

[44]  Theodor D. Sterling,et al.  Publication decisions revisited: the effect of the outcome of statistical tests on the decision to p , 1995 .

[45]  Jacob Cohen The earth is round (p < .05) , 1994 .

[46]  V. Pollock,et al.  Meta-analytic synthesis for theory development. , 1994 .

[47]  L. Daston,et al.  The Image of Objectivity , 1992 .

[48]  Harry T. Reis,et al.  Publication Trends in JPSP: A Three-Decade Review , 1992 .

[49]  Gideon Engler,et al.  AESTHETICS IN SCIENCE AND IN ART , 1990 .

[50]  John T. Cacioppo,et al.  The Elaboration Likelihood Model of Persuasion , 1986, Advances in Experimental Social Psychology.

[51]  Susan Leigh Star,et al.  Changing Order: Replication and Induction in Scientific Practice by H. M. Collins (review) , 1988, Technology and Culture.

[52]  Gordon H. Bower,et al.  Failure to replicate mood-dependent retrieval , 1985 .

[53]  Paul T. P. Wong,et al.  Implicit editorial policies and the integrity of psychology as an empirical science. , 1981 .

[54]  S. Chaiken Heuristic versus systematic information processing and the use of source versus message cues in persuasion. , 1980 .

[55]  Seymour Epstein,et al.  The stability of behavior: II. Implications for psychological research. , 1980 .

[56]  F. Samelson J. B. Watson's Little Albert, Cyril Burt's twins, and the need for a critical science. , 1980 .

[57]  O. Gillie Did Sir Cyril Burt Fake His Research on Heritability of Intelligence? Part I. , 1977 .

[58]  A. C. Elms The crisis of confidence in social psychology. , 1975 .

[59]  A. Greenwald Consequences of Prejudice Against the Null Hypothesis , 1975 .

[60]  A. W. Wicker Attitudes Versus Actions: The Relationship of Verbal and Overt Behavioral Responses to Attitude Objects. , 1969 .

[61]  R. Rosenthal,et al.  Artifact in behavioral research , 1969 .

[62]  D. Lykken Statistical significance in psychological research. , 1968, Psychological bulletin.

[63]  A. Sapolsky AN EFFORT AT STUDYING RORSCHACH CONTENT SYMBOLISM: THE FROG RESPONSE. , 1964, Journal of consulting psychology.

[64]  Paul Adrien Maurice Dirac,et al.  The Evolution of the Physicist's Picture of Nature , 1963 .