WHERE DOES THE BALANCE OF EVIDENCE LIE WITH RESPECT TO CONFIDENCE ?

The primary ai m of most se quential sampling model s of discrimination has been to explain subjects' abilit y to trade speed for accuracy. Less commonly, such model s have attempted to account for the confidence with which response s are made. Results are reviewed from some half dozen studies, conforming to an expanded judgmen t paradigm, in which accuracy, response time and confidence were measured . Out of several suggested theoretical bases for confidence, the balance-of-evidence hypothesi s appears to be most successful. However, the results also underl ine the importance of assumptions regarding the process in which internal representations of confidence are converted into overt responses . In 1923, when Clark Trow wrote to J.B. Watson to ask wha t was the behavi ourist position on confiden ce, Watson replied “I am afraid you have come t o the wrong market” . Despite this, and even i f we excl ude self-confidence, the topic still has a considerabl e market currency. We depend on assessment s of confidence to make investment choices, to evaluat e the credibilit y of eyewitness testimony, and to carry out (or undergo) surgical procedures. In psychology, confidence measure s have been relied on for over a centur y to test hypothese s about perception, memor y and decision making both directly and through th e construction of receiver operating char acteristic (ROC) curves for signal detection theory analyses. What is remarkabl e is that , despite its practical importance and pervasivene ss, the variable of confidence seems to have played a Cinderella role i n cognitive psychology – relied on for its usefulness , but overlooked as an interesting variabl e in its own right. A TAXONOMY OF CONFIDENCE EFFECTS Before looking at alternative formulations for confidence, it is useful to outline a possible taxonomy of confidence effects. First, we need to distinguish between different kinds of task. Assuming that we restrict ourselves to perceptual judgments , we might then want to treat discriminati on, identifi cation, and det ection separately . We might also want to distinguish between tasks involving different number s of response alternatives. The next set of distinctions concerns the way a task is implemented. For example, a familiar sensor y discrimination would be an exper iment in which the subject is shown a pai r of lines and has to decide which line is the longer. So-called expanded judgments (EJs) are less familiar. On each trial of an EJ task, the subject has the opportunity to inspect multiple stimulus elements a nd to make a j udgment concerning either the sample presente d or the population from which the sample i s drawn. The name “expanded judgment ” means that the task attempts to externalise the hypothesised , interior process of sequential sampling in a psychophysi cal judgment . (In earlier versions, this meant expanding the proces in time.) For example, in a temporal EJ task, using discrete stimulus elements , the subject may be presente d with a sequence of flashes on one or the other of two lamps, with instructions to decide which lamp i s set to flash mor e frequently. By comparison, in a task using continuously varying elements , the subject might inspect a succession of horizontal line segments , extending to the right or the left of a central , vertical line, represen ting zero. The subject is told that the sequence is generate d by drawing from a normal distribution, with negative number s bei ng represented by leftward -extendi ng segment s and positive number s by rightward-extendi ng segments . On a given trial, the task is to decide whether the sample (o r the distribution), used to gener ate the segments , has a mean that is positive or negative. In spatial EJ tasks, the element s are distributed spatially rather than temporally. As with sensory judgments , EJ tasks ma y conform to either a timeor an informationlimited paradigm. For example, in a time-limited, discrete, temporal EJ task, the subject may be presente d with a sequence – of fixed length of left or right flashes, and be asked to decide whether that sampl e has more right or more left flashes. In an information-limited version, subjects are allowed to continue inspecting flashes until they decide whether the population from which the flashes are drawn has more right or more left flashes. In an information-limited EJ discrimination task, there are at least four distinct variables that affect response probability , time, and confidence. These are: (1) the discriminability of the two set s of stimulus elements ; (2) the speed-accuracy tradeoff (or inferred degree of caution for both responses) , adopted by the subject; (3) the relative degree of caution voluntarily exercised fo r one response in preference to the other ; and (4) the conscious expectation, held by the subject, that one or the other response is more likely. Each of these variables can be mani pulated in various ways. However, t he most extensivel y studied variabl e is that of discriminability. In the case o f a discrete, temporal EJ task, this would be mani pulated by varying the relative frequency of the two binary stimulus elements . In tasks empl oying continuously varying s timulus elements , the situation is a littl e richer. Assuming that the stimulus elements ar e normally distributed, their discriminability can be mani pulated in three mai n ways: (1) varying the mean, m, while holding the standard deviation, s, constant; (2) varying both m and s; or (3) varying s, while holding m constant. Each manipulation of discriminability can be examined for its direct effects on respons e probability , time, and confidence, and on the interrelations between these variables. HYPOTHESES REGARDING CONFIDENCE Confidence obeys three genera lisations: (1) it is a direct function of discriminability; (2) it is a direct function of accuracy; and (3) it varies inversel y with respons e time. Peirce and Jastrow (1884) first quantified the second genera lisation in the descriptive formula, C = h log (p/1-p), where C represent s the measure d degree of confidence, p denotes the probabilit y of a response bei ng correct, and h is a constant . Later, Volkmann (1934) tried to capture the third, using the equation, C = 0.5(a/t-b) + 0.5, where t is time and a and b are constants . A century after Peirce’s first article, three accounts appeared, each incorporating confidence into a theoretical model of the discrimination process. In one, Ratcli ff (1978) proposed a diffusion model in which two conf licting evidence streams continuousl y drive a random walk towards one or the other of two thresholds . The rate of drift of the walk is determined by the discriminability of the alternatives, while the thres holds are assumed to be set by the subject. Because the only information about discriminability is the decisi on time, Ratcli ff proposed that confidence be an inverse function of the actual time taken by a subject. A similar proposal was put forward by Link and Heath (1975). In their random walk model , it is differences between alternative stimulus inputs and an interna l referent that ar e used to drive the wal k towards an upper threshold, A, or a lower threshold, -A. In addition, these differences are input to the walk at discret e intervals, rather than continuously. In Link and Heath’s model, confidence is postulated to be a function of the distance (A-O) traversed by the walk, multiplied by a discriminability parameter , θ, where θ is evaluated in terms of the parameter s (m, s) of the distribution of sampled differences, and O is the star ting position of the walk (Heath, 1984; see also Vickers & Smith, 1985). The third model is the accumulator, suggested by Vickers (1979). In this model, stimulus differences are sampled at discret e intervals, with positive and negative differences being accumulate d in two separat e total s until one or the other reaches a preset threshold. On this model, confidence is deter mined by the balance-of-evidence (i.e., by the difference between the two totals at the moment a decisi on is reached or sampling terminated. Recently, Juslin and Ol sson (1997) proposed a window-sampling model of sensor y discrimination. In this model, discriminal differences are sampled , one at a time, and average d over a moving window. Confidence is deter mined by the ratio of sampled differences, in favour of the successful response, that are presen t in the window when a response is made. EMPIRICAL COMPARISONS BETWEEN ALTERNATIVE HYPOTHESES I should like to compar e these hypothese s about confidence. In particular (though not exclusively), I shall examine thei r accounts of the results from some hal f dozen EJ studies, carried out over the last three years . The experimental details are summarised in Tabl e 1. I shall focus on specific features , moving sequen tially (and sel ectively) through the taxonomy. Table 1. Summary of expanded judgment tasks