The pseudodiagnosticity trap: Should participants consider alternative hypotheses?

The pseudodiagnosticity task has been used as an example of the tendency on the part of participants to incorrectly assess Bayesian constraints in assessing data, and as a failure to consider alternative hypotheses in a probabilistic inference task. In the task, participants are given one value, the anchor value, corresponding to P(D1|H) and may choose one other value, either P(D1|¬!H), P(D2|H), or P(D2|not;!H). Most participants select P(D2|H), or P(D2|¬!H) which have been considered inappropriate (and called pseudodiagnostic) because only P(D1|¬!H) allows use of Bayes' theorem. We present a new analysis based on probability intervals and show that selection of either P(D2|H), or P(D2|¬!H) is in fact pseudodiagnostic, whereas choice of P(D1|¬!H) is diagnostic. Our analysis shows that choice of the pseudodiagnostic values actually increases uncertainty regarding the posterior probability of H, supporting the original interpretation of the experimental findings on the pseudodiagnosticity task. The argument illuminates the general proposition that evolutionarily adaptive heuristics for Bayesian inference can be misled in some task situations.

[1]  Jonathan D. Nelson.  Finding useful questions: on Bayesian diagnosticity, probability, impact, and information gain. , 2005, Psychological review.

[2]  N. Chater,et al.  Rational models of cognition , 1998 .

[3]  P. Johnson-Laird,et al.  Psychology of Reasoning: Structure and Content , 1972 .

[4]  M. Cowles.  Statistics in Psychology: An Historical Perspective , 1989 .

[5]  B P Wood,et al.  Decision making in radiology. , 1999, Radiology.

[6]  N. Chater,et al.  Conditional inference and constraint satisfaction: Reconciling mental models and the probabilistic approach? , 2010 .

[7]  N. Chater,et al.  An introduction to rational models of cognition , 1998 .

[8]  Niki Pfeifer,et al.  Framing human inference by coherence based probability logic , 2009, J. Appl. Log..

[9]  S. Stigler,et al.  The History of Statistics: The Measurement of Uncertainty before 1900 by Stephen M. Stigler (review) , 1986, Technology and Culture.

[10]  H Garavan,et al.  On people’s understanding of the diagnostic implications of probabilistic data , 1996, Memory & cognition.

[11]  John Aitchison,et al.  Statistical Prediction Analysis , 1975 .

[12]  Joseph L. Austerweil,et al.  A rational analysis of confirmation with deterministic hypotheses , 2008 .

[13]  M E Doherty,et al.  'Pseudodiagnosticity' in an idealized medical problem-solving environment. , 1982, Journal of medical education.

[14]  Clifford R. Mynatt,et al.  The influence of feedback and diagnostic data on pseudodiagnosticity , 1981 .

[15]  Niki Pfeifer,et al.  The conditional in mental probability logic , 2010 .

[16]  J. Klayman,et al.  Confirmation, Disconfirmation, and Informa-tion in Hypothesis Testing , 1987 .

[17]  R. Skov,et al.  Information-gathering processes: Diagnosticity, hypothesis-confirmatory strategies, and perceived hypothesis confirmation. , 1986 .

[18]  S. Sloman,et al.  The probability of causal conditionals , 2007, Cognitive Psychology.

[19]  Michael E. Doherty,et al.  Information Relevance, Working Memory, and the Consideration of Alternatives , 1993 .

[20]  N. Chater,et al.  Rational models of cognition , 1998 .

[21]  Vincenzo Crupi,et al.  Pseudodiagnosticity revisited. , 2009, Psychological review.