Confirmation, Disconfirmation, and Informa-tion in Hypothesis Testing

Strategies for hypothesis testing in scientific investigation and everyday reasoning have interested both psychologists and philosophers. A number of these scholars stress the importance of disconfir. marion in reasoning and suggest that people are instead prone to a general deleterious "confirmation bias" In particula~ it is suggested that people tend to test those cases that have the best chance of verifying current beliefs rather than those that have the best chance of falsifying them. We show, howeve~ that many phenomena labeled "confirmation bias" are better understood in terms of a general positive test strate~. With this strategy, there is a tendency to test cases that are expected (or known) to have the property of interest rather than those expected (or known) to lack that property. This strategy is not equivalent to confirmation bias in the first sense; we show that the positive test strategy can be a very good heuristic for determining the truth or falsity of a hypothesis under realistic conditions~ It can, howeve~ lead to systematic errors or inefficiencies. The appropriateness of human hypotheses-testing strategies and prescriptions about optimal strategies must he understood in terms of the interaction between the strategy and the task at hand.

[1]  S. Amano,et al.  Psychology of Reasoning , 1923 .

[2]  Jerome S. Bruner,et al.  Personality dynamics and the process of perceiving. , 1951 .

[3]  R. R. Blake,et al.  Perception : an approach to personality , 1951 .

[4]  J. Bruner,et al.  A study of thinking , 1956 .

[5]  William M. Smith,et al.  A Study of Thinking , 1956 .

[6]  M. Kendall,et al.  The Logic of Scientific Discovery. , 1959 .

[7]  P. Wason.  On the Failure to Eliminate Hypotheses in a Conceptual Task , 1960 .

[8]  N. Wetherick,et al.  Eliminative and Enumerative Behaviour in a Conceptual Task , 1962 .

[9]  P. C. Wason.  Reply to Wetherick , 1962 .

[10]  J. Smedslund.  THE CONCEPT OF CORRELATION IN ADULTS , 1963 .

[11]  H. M. Jenkins,et al.  The display of information and the judgment of contingency. , 1965, Canadian journal of psychology.

[12]  H. M. Jenkins,et al.  JUDGMENT OF CONTINGENCY BETWEEN RESPONSES AND OUTCOMES. , 1965, Psychological monographs.

[13]  W. Edwards.  Optimal strategies for seeking information: Models for statistics, choice reaction times, and human information processing ☆ , 1965 .

[14]  B. Foss.  New Horizons in Psychology 1 , 1966 .

[15]  W. Edwards,et al.  Conservatism in a simple probability inference task. , 1966, Journal of experimental psychology.

[16]  M. Levine.  Hypothesis behavior by humans during discrimination learning. , 1966, Journal of experimental psychology.

[17]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[18]  P. Wason.  >On the Failure to Eliminate Hypotheses...< – A Second Look , 1968 .

[19]  T. Trabasso.  Attention in learning , 1968 .

[20]  L E Bourne,et al.  Learning conceptual rules. II. The role of positive and negative instances. , 1968, Journal of experimental psychology.

[21]  Ian I. Mitroff.  The Subjective Side of Science , 1969 .

[22]  I. Lakatos,et al.  Criticism and the Growth of Knowledge: Falsification and the Methodology of Scientific Research Programmes , 1970 .

[23]  Willem J. M. Levelt,et al.  Advances in psycholinguistics , 1970 .

[24]  M. Levine.  Human discrimination learning: the subset-sampling assumption. , 1970, Psychological bulletin.

[25]  P. Johnson-Laird,et al.  Psychology of Reasoning: Structure and Content , 1972 .

[26]  Kathryn T. Spoehr,et al.  The direct measurement of hypothesis-sampling strategies , 1973 .

[27]  Z. Annau,et al.  An operant discrimination task allowing variability of reinforced response patterning. , 1973, Journal of the experimental analysis of behavior.

[28]  J. S. Evans,et al.  Matching bias in the selection task. , 1973 .

[29]  Herbert A. Simon,et al.  Does Scientific Discovery Have a Logic , 1973 .

[30]  M. Sachs.  Objective Knowledge , 1974 .

[31]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[32]  L. E. Bourne,et al.  An inference model for conceptual rule learning. , 1974 .

[33]  J E Taplin.  Evaluation of hypotheses in concept identification , 1975, Memory & cognition.

[34]  R. L. Solso.  Theories in cognitive psychology : the Loyola symposium , 1975 .

[35]  I. Lakatos.  Falsification and the Methodology of Scientific Research Programmes , 1976 .

[36]  Joseph P. Martino,et al.  The subjective side of science : Mitroff, Ian I., Elsevier Scientific Publishing Co., Amsterdam, 1974, 329 pp., $11.50 , 1976 .

[37]  Clifford R. Mynatt,et al.  Confirmation Bias in a Simulated Research Environment: An Experimental Study of Scientific Inference , 1977 .

[38]  W. Kintsch,et al.  Memory and cognition , 1977 .

[39]  W. D. Scipio.  Scientist As Subject: The Psychological Imperative , 1978 .

[40]  Clifford R. Mynatt,et al.  Consequences of Confirmation and Disconfirmation in a Simulated Research Environment , 1978 .

[41]  C. Gettys,et al.  Hypothesis Generation and Plausibility Assessment. , 1978 .

[42]  W. Swann,et al.  Hypothesis-Testing Processes in Social Interaction , 1978 .

[43]  R. Hogarth,et al.  Confidence in judgment: Persistence of the illusion of validity. , 1978 .

[44]  C. Gettys,et al.  Hypothesis plausibility and hypothesis generation , 1979 .

[45]  L. Ross,et al.  Biased Assimilation and Attitude Polarization: The Effects of Prior Theories on Subsequently Considered Evidence , 1979 .

[46]  H. J. Einhorn,et al.  Cognitive processes in choice and decision behavior , 1979 .

[47]  Ryan D. Tweney,et al.  Strategies of Rule Discovery in an Inference Task , 1980 .

[48]  Rationality and Disconfirmation , 1980 .

[49]  H. Shaklee,et al.  A rule analysis of judgments of covariation between events , 1980, Memory & cognition.

[50]  L. Ross,et al.  Human Inference: Strategies and Shortcomings of Social Judgment. , 1981 .

[51]  Rationality and Authority: On the Confusion of Justification and Permission , 1980 .

[52]  J. Fox.  Making Decisions under the Influence of Memory. , 1980 .

[53]  M. Fishbein.  Progress in social psychology , 1980 .

[54]  J. E. Tschirgi,et al.  Sensible reasoning: A hypothesis about hypotheses. , 1980 .

[55]  R. Hogarth.  Beyond discrete biases: Functional and dysfunctional aspects of judgmental heuristics. , 1981 .

[56]  Harriet Shaklee,et al.  Development of Rule Use in Judgments of Covariation between Events. , 1981 .

[57]  R. Sternberg,et al.  Evaluation of evidence in causal inference. , 1981 .

[58]  Jennifer Crocker,et al.  Judgment of Covariation by Social Perceivers , 1981 .

[59]  B. Schwartz.  Control Of Complex, Sequential Operants By Systematic Visual Information In Pigeons , 1981 .

[60]  R. Michels,et al.  Diagnostic and Statistical Manual of Mental Disorders, 3rd ed , 1981 .

[61]  Mark Snyder,et al.  Seek, and ye shall find: Testing hypotheses about other people: The Ontario Symposium , 1981 .

[62]  Y. Trope,et al.  Confirmatory and diagnosing strategies in social information gathering. , 1982 .

[63]  Barry Schwartz,et al.  Reinforcement-Induced Behavioral Stereotypy: How Not To Teach People To Discover Rules , 1982 .

[64]  Hillel J. Einhorn,et al.  Judgment under uncertainty: Learning from experience and suboptimal rules in decision making , 1982 .

[65]  A Cross-Study Analysis of Covariation Judgments. , 1982 .

[66]  Harriet Shaklee,et al.  Sources of error in judging event covariations: Effects of memory demands. , 1982 .

[67]  Leslie Helen Kern.  The effect of data error in inducing confirmatory inference strategies in scientific hypothesis testing , 1982 .

[68]  Stephen A. Yachanin,et al.  The effect of thematic content on cognitive strategies in the four-card selection task , 1982 .

[69]  J. Crocker.  Biased Questions in Judgment of Covariation Studies , 1982 .

[70]  Richard A. Griggs,et al.  The elusive thematic‐materials effect in Wason's selection task , 1982 .

[71]  A. Tversky,et al.  Causal Schemata in Judgments under Uncertainty , 1982 .

[72]  G. A. Miller,et al.  Book Review Nisbett, R. , & Ross, L.Human inference: Strategies and shortcomings of social judgment.Englewood Cliffs, N.J.: Prentice-Hall, 1980. , 1982 .

[73]  Daniel M. Wegner,et al.  Where Leading Questions Can Lead: The Power of Conjecture in Social Interaction , 1982 .

[74]  Ward Edwards,et al.  Judgment under uncertainty: Conservatism in human information processing , 1982 .

[75]  Richard A. Griggs,et al.  The effects of experience on performance in Wason’s selection task , 1982, Memory & cognition.

[76]  Clifford R. Mynatt,et al.  Rationality and Disconfirmation: Further Evidence , 1982 .

[77]  Stephen J. Hoch,et al.  Cue redundancy and extra logical inferences in a deductive reasoning task , 1983, Memory & cognition.

[78]  Richard A. Griggs,et al.  The Effects of Problem Content and Negation on Wason's Selection Task , 1983 .

[79]  Hal R. Arkes,et al.  Estimates of contingency between two dichotomous variables. , 1983 .

[80]  B. Fischhoff,et al.  Hypothesis Evaluation from a Bayesian Perspective. , 1983 .

[81]  J. Darley,et al.  A hypothesis-confirming bias in labeling effects. , 1983 .

[82]  Counselor Hypothesis-Testing Strategies. , 1983 .

[83]  Yaacov Trope,et al.  Information-gathering strategies in hypothesis-testing , 1983 .

[84]  Martin D. S. Braine,et al.  Conversational comprehension processes are responsible for reasoning fallacies in children as well as adults: If is not the biconditional. , 1983 .

[85]  R. Duda,et al.  Expert Systems Research. , 1983, Science.

[86]  Yaacov Trope,et al.  The questions lay interviewers ask , 1984 .

[87]  L. Alloy,et al.  Assessment of covariation by humans and animals: the joint influence of prior expectations and current situational information. , 1984, Psychological review.

[88]  L. Alloy,et al.  Assessment of covariation by humans and animals: The joint influence of prior expectations and current situational information. , 1984 .

[89]  M. Gorman,et al.  A Comparison of Disconfirmatory, Confirmatory and Control Strategies on Wason's 2–4–6 Task , 1984 .

[90]  W. Swann,et al.  Quest for accuracy in person perception: a matter of pragmatics. , 1984, Psychological review.

[91]  Jonathan St. B. T. Evans,et al.  Thinking and reasoning : psychological approaches , 1985 .

[92]  J. E. Tschirgi,et al.  Logical knowledge and cue redundancy in deductive reasoning , 1985, Memory & cognition.

[93]  Studies in the History of Psychology and the Social Sciences 3 , 1985 .

[94]  R. Tweney.  Faraday’s Discovery of Induction: A Cognitive Approach , 1985 .

[95]  R. Hogarth,et al.  Judging probable cause. , 1986 .

[96]  Stephen J. Hoch,et al.  Consumer Learning: Advertising and the Ambiguity of Product Experience , 1986 .

[97]  Michael E. Gorman,et al.  How the possibility of error affects falsification on a task that models scientific problem solving , 1986 .

[98]  Guy Politzer,et al.  Laws of language use and formal logic , 1986 .

[99]  R. Skov,et al.  Information-gathering processes: Diagnosticity, hypothesis-confirmatory strategies, and perceived hypothesis confirmation. , 1986 .

[100]  H. Brachinger,et al.  Decision analysis , 1997 .

[101]  William B. Swann,et al.  Confirmatory Search Strategies in Social Interaction: How, When, Why, and With What Consequences , 1987 .

[102]  Gary James Jason,et al.  The Logic of Scientific Discovery , 1988 .