In What Sense is P(A|B) P(B) = P(A,B)? The Relationship between Distributional Format and Subjective Probability Estimates

In What Sense is P(A|B) P(B) = P(A,B)? The Relationship between Distributional Format and Subjective Probability Estimates Belinda Bruza, Matthew B. Welsh, Daniel J. Navarro & Stephen H. Begg ({belinda.bruza, matthew.welsh, daniel.navarro, steve.begg}@adelaide.edu.au) The University of Adelaide, SA 5005, Australia Abstract Problem Representation The elicitation of uncertainty is a topic of interest in a range of disciplines. The conversion of expert beliefs into probability distributions can play a role in assisting key decisions in industry. However, elicitation methods can be prone to bias. In this paper we investigate the effect of changing the presentation of stimulus information and question format on elicited judgments of marginal, conditional and joint probabilities. Participants taught a probability distribution in one structure were expected to have difficulty assessing the distribution in another structure. While this pattern was not found, it turned out that training participants on the more difficult task (learning from a conditional structure) improved overall performance. A consistent finding in the decision-making literature is that people are sensitive to the surface representation of a problem. For instance: options described in terms of gains are evaluated differently to the same options when described in terms of losses (Kahneman & Tversky, 1979); changing the surface form of the Tower of Hanoi problem can alter the difficulty of the task (Gunzelmann & Blessing, 2000); and statistical problems expressed in terms of frequencies seem to be easier than the same problems described in terms of probabilities (Gigerenzer & Hoffrage, 1995). One interesting variation on the question of problem representation arises when people need to learn about and report on the joint distribution of two variables, A and B. Mathematically, we can describe the distribution to be learned and subsequently elicited in three formally equivalent ways, by noting that: Keywords: decision making; cognitive biases; elicitation; probability learning The “elicitation of uncertainty” is a general term that is often used to refer to methods for translating a set of implicit beliefs into an explicit probability distribution (Wolfson, 2001). The reason for using these methods is to allow researchers to incorporate subjective expert knowledge into a quantitative model that makes predictions about future events (Morgan & Keith, 1995). In view of this, good elicitation methods can play an important role in guiding decision making in a range of industries in which uncertain outcomes are central. One of the main impediments to widespread use of elicitation techniques in applied settings is the inherent difficulty of the task. This difficulty is caused by the many well-known decision-making heuristics and biases, which can distort the estimates of the underlying beliefs. For instance, anchoring and adjustment, representativeness, availability, base rate neglect and overconfidence (see Tversky & Kahneman, 1974; Bar- Hillel, 1980; Lichtenstein, Fischoff, & Phillips, 1982) have all been found to influence the judgments people make in an elicitation context, in both lay and expert populations (see, e.g., Eddy, 1982; Welsh, Bratvold & Begg, 2005). Moreover, people often mistake conditional probabilities for joint probabilities (Pollasek et al., 1987) since these are easier to compute (Lewis & Keren, 1999), and often experience difficulties with characterizing the conditioning event (Bar-Hillel & Falk, 1982). People may confuse one conditional probability P(A | B) with another P(B | A), or have difficulties interpreting instructions related to probability (Bar-Hillel, 1980; Fiedler et al., P(A, B) = P(A | B) P(B) = P(B | A) P(A) For the current purposes we refer to each of these three variations as a “problem format”, and note that while all three formats describe to the same distribution over A and B, there is no guarantee that people will treat them as such. Indeed, in view of the known differences in how people estimate marginal probabilities, conditional probabilities and joint probabilities, we would expect to observe fairly substantial differences between formats. In this paper we describe an experiment that examines (1) whether one format for the problem leads to superior learning and subsequent probability estimation in general, and (2) whether learning in one format makes it easier to report on questions framed in the same format. Should either of these two effects be observed, a natural method for improving elicitation in an applied context would be to alter the presentation format to be more suited to the expectations of the expert whose beliefs are to be elicited. Method Participants Participants were 60 students (18 male) studying at the University of Adelaide, aged 18 to 37 years, and were paid $15 for their time.