Biases to the left, fallacies to the right: stuck in the middle with null hypothesis significance testing

Krueger (1998) argues that biases are simple to find because of the ease of disproving overly specific null hypotheses of normative behavior. We support his argument with examples of biases falling on both sides of the normative null hypothesis in behavioral decision making, and we show how a broader definition of normativity can lead to opposite conclusions about human rationality. To overcome the problems of asymmetric hypothesis testing -- encouraging the discovery of biases and discouraging the specification of particular cognitive processes -- we recommend two steps: submit precisely specified theories to symmetrical tests, and model sound reasoning based on problem-specific psychological assumptions.

[1]  Peter M. Todd,et al.  Testing the ecological rationality of base rate neglect , 2002 .

[2]  John Cohen,et al.  A CONFIRMATION OF THE INERTIAL‐ψ EFFECT IN SEQUENTIAL CHOICE AND DECISION , 1972 .

[3]  Gerd Gigerenzer,et al.  Psychological Challenges for Normative Models , 1998 .

[4]  D Kahneman,et al.  On the reality of cognitive illusions. , 1996, Psychological review.

[5]  C. E. M. Hansel,et al.  The nature of decisions in gambling: Equivalence of single and compound subjective probabilities , 1957 .

[6]  M. Birnbaum.  Base Rates in Bayesian Inference: Signal Detection Analysis of the Cab Problem , 1983 .

[7]  N. Cowan,et al.  A generalized signal detection model to predict rational variation in base rate use , 1999, Cognition.

[8]  Ralph Hertwig,et al.  Many Reasons or Just One: How Response Mode Affects Reasoning in the Conjunction Problem , 1998 .

[9]  Z. J. Ulehla,et al.  Internal consistency of subjective probabilities. , 1965, Journal of Experimental Psychology.

[10]  L. Beach,et al.  Man as an Intuitive Statistician , 2022 .

[11]  W. A. Wagenaar.  Generation of random sequences by human subjects: A critical survey of literature. , 1972 .

[12]  A. Tversky,et al.  The hot hand in basketball: On the misperception of random sequences , 1985, Cognitive Psychology.

[13]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[14]  Gerd Gigerenzer,et al.  How to Improve Bayesian Reasoning Without Instruction: Frequency Formats , 1995 .

[15]  G. Gigerenzer.  On Narrow Norms and Vague Heuristics: A Reply to Kahneman and Tversky (1996) , 1996 .

[16]  J. Krueger.  The bet on bias: A foregone conclusion? , 1998 .

[17]  A. Tversky,et al.  Subjective Probability: A Judgment of Representativeness , 1972 .

[18]  Gerd Gigerenzer,et al.  The "conjunction fallacy" revisited : How intelligent inferences look like reasoning errors , 1999 .