The Influence of Prior Beliefs on Scientific Judgments of Evidence Quality

This paper is concerned with the influence of scientists’ prior beliefs on their judgments of evidence quality. A laboratory experiment using advanced graduate students in the sciences (study 1) and an experimental survey of practicing scientists on opposite sides of a controversial issue (study 2) revealed agreement effects. Research reports that agreed with scientists’ prior beliefs were judged to be of higher quality than those that disagreed. In study 1, a prior belief strength X agreement interaction was found, indicating that the agreement effect was larger for general, evaluative judgments than for more specific, analytical judgments. A Bayesian analysis indicates that the pattern of agreement effects found in these studies may be normatively defensible, although arguments against implementing a Bayesian approach to scientific judgment are also advanced.

[1]  L. Festinger,et al.  When Prophecy Fails , 1956 .

[2]  S. Oskamp.  OVERCONFIDENCE IN CASE-STUDY JUDGMENTS. , 1965, Journal of consulting psychology.

[3]  G. Pitz,et al.  Sequential effects in the revision of subjective probabilities. , 1967, Canadian journal of psychology.

[4]  R. Kleck,et al.  Dogmatism and responses to opinion-consistent and opinion-inconsistent information. , 1967, Journal of personality and social psychology.

[5]  Gordon F. Pitz,et al.  Confidence and decision speed in the revision of opinion , 1968 .

[6]  Ian I. Mitroff,et al.  The Subjective Side of Science , 1969 .

[7]  R. Kirk.  Experimental Design: Procedures for the Behavioral Sciences , 1970 .

[8]  R. Merton.  The Normative Structure of Science , 1973 .

[9]  Investigating the paranormal , 2010, Nature.

[10]  C. Batson.  Rational processing or rationalization? The effect of disconfirming information on a stated religious belief. , 1975 .

[11]  Timothy D. Wilson,et al.  Telling more than we can know: Verbal reports on mental processes. , 1977 .

[12]  W. D. Scipio.  Scientist As Subject: The Psychological Imperative , 1978 .

[13]  D. Butler,et al.  The Scientific Credibility of ESP , 1978 .

[14]  M. Snyder,et al.  Reconstructing the Past: Some Cognitive Consequences of Person Perception , 1978 .

[15]  L. Ross,et al.  Biased Assimilation and Attitude Polarization: The Effects of Prior Theories on Subsequently Considered Evidence , 1979 .

[16]  L. Ross,et al.  Human Inference: Strategies and Shortcomings of Social Judgment. , 1981 .

[17]  C. Hansel.  ESP and parapsychology: A critical reevaluation , 1980 .

[18]  B. Fischhoff,et al.  Reasons for confidence. , 1980 .

[19]  J. Alcock,et al.  Critical Thinking and Belief in the Paranormal , 1980 .

[20]  Clayton Koelb.  A Psychometric Study of Literary-Critical judgment , 1982 .

[21]  W. H. Jones,et al.  Anomalistic psychology: A study of extraordinary phenomena of behavior and experience , 1982 .

[22]  G. Nigel Gilbert,et al.  Accounting for Error: How Scientists Construct their Social World when they Account for Correct and Incorrect Belief , 1982 .

[23]  M. Lepper,et al.  Considering the opposite: a corrective strategy for social judgment. , 1984, Journal of personality and social psychology.

[24]  C. Honorton.  Meta-analysis of psi ganzfeld research: A response to Hyman. , 1985 .

[25]  Irvin L. Child,et al.  Psychology and anomalous observations: The question of ESP in dreams. , 1985 .

[26]  J. Alcock,et al.  Parapsychology: Science of the anomalous or search for the soul? , 1987 .

[27]  David Faust,et al.  Eliminating the hindsight bias. , 1988 .

[28]  J. Baron,et al.  Outcome bias in decision evaluation. , 1988, Journal of personality and social psychology.

[29]  D. Kuhn,et al.  The development of scientific thinking skills , 1988 .

[30]  Jonathan J. Koehler,et al.  Veridical Verdicts: Increasing Verdict Accuracy Through the Use of Overtly Probabilistic Evidence and Methods , 1990 .

[31]  Craig A. Anderson,et al.  Belief Perseverance, Biased Assimilation, and Covariation Detection: The Effects of Hypothetical Social Theories and New Data , 1992 .