Type 2 tasks in the theory of signal detectability: Discrimination between correct and incorrect decisions

It has been known for over 40 years that there are two fundamentally different kinds of detection tasks in the theory of signal detectability. The Type 1 task is to distinguish between events defined independently of the observer; the Type 2 task is to distinguish between one’s own correct and incorrect decisions about those Type 1 events. For the Type 1 task, the behavior of the detector can be summarized by the traditional receiver operating characteristic (ROC) curve. This curve can be compared with a theoretical ROC curve, which can be generated from overlapping probability functions conditional on the Type 1 events on an appropriate decision axis. We show how to derive the probability functions underlying Type 2 decisions from those for the Type 1 task. ROC curves and the usual measures of performance are readily obtained from those Type 2 functions, and some relationships among various Type 1 and Type 2 performance measures are presented. We discuss the relationshiPbetween Type 1 and Type 2 confidence ratings and caution against the practice of presenting transformed Type 2 ratings as empirical Type 1 ratings.

[1]  J. Jastrow,et al.  On small differences in sensation , 1884 .

[2]  J. Wolfowitz,et al.  Introduction to the Theory of Statistics. , 1951 .

[3]  W. W. Peterson,et al.  The theory of signal detectability , 1954, Trans. IRE Prof. Group Inf. Theory.

[4]  James P. Egan.  On the Transmission and Confirmation of Messages in Noise , 1956 .

[5]  T. Marill.  Detection theory and psychophysics , 1956 .

[6]  James P. Egan,et al.  Source and Receiver Behavior in the Use of a Criterion , 1956 .

[7]  Irwin Pollack,et al.  Confidence Ratings, Message‐Reception, and the Receiver Operator Characteristic , 1957 .

[8]  Irwin Pollack,et al.  On Indices of Signal and Response Discriminability , 1959 .

[9]  T. Birdsall,et al.  Two Types of ROC Curves and Definitions of Parameters , 1959 .

[10]  J SWETS,et al.  Decision processes in perception. , 1961, Psychological review.

[11]  B. L. Welch,et al.  Introduction to the Theory of Statistics , 1964 .

[12]  B. Murdock.  The criterion problem in short-term memory. , 1966, Journal of experimental psychology.

[13]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[14]  Donald A. Norman,et al.  Strength theory of decision rules and latency in retrieval from short-term memory , 1969 .

[15]  William P. Banks,et al.  Signal detection theory and human memory. , 1970 .

[16]  J. Swets.  The Relative Operating Characteristic in Psychology , 1973, Science.

[17]  Alice F. Healy,et al.  Criterion shifts in recall. , 1973 .

[18]  Franklin A. Graybill,et al.  Introduction to the Theory of Statistics, 3rd ed. , 1974 .

[19]  B. Murdock.  Human memory: Theory and data. , 1975 .

[20]  James P. Egan,et al.  Signal detection theory and ROC analysis , 1975 .

[21]  D. Vickers.  Decision processes in visual perception , 1979 .

[22]  William R. Ferrell,et al.  A model of calibration for subjective probabilities , 1980 .

[23]  Gary Cantor,et al.  On the Relationship Between Implicit and Explicit Modes in the Learning of a Complex Rule Structure , 1980 .

[24]  W. Ferrell,et al.  Detectability of correctness: A measure of knowing that one knows , 1982 .

[25]  B. Fischhoff,et al.  Calibration of probabilities: the state of the art to 1980 , 1982 .

[26]  G. Keren,et al.  On the ability of monitoring non-veridical perceptions and uncertain knowledge: some calibration studies. , 1988, Acta psychologica.

[27]  Neil A. Macmillan,et al.  Detection Theory: A User's Guide , 1991 .

[28]  G. Gigerenzer,et al.  Probabilistic mental models: a Brunswikian theory of confidence. , 1991, Psychological review.

[29]  G. Keren.  Calibration and probability judgements: Conceptual and methodological issues , 1991 .

[30]  R. Boven,et al.  Reduction of unique noise in the psychophysics of hearing by Group Operating Characteristic analysis. , 1991, Psychological bulletin.

[31]  T. Critchfield.  Signal-detection properties of verbal self-reports. , 1993, Journal of the experimental analysis of behavior.

[32]  Jochen Braun,et al.  Blindsight in normal observers , 1995, Nature.

[33]  R. Duncan Luce,et al.  Several unresolved conceptual problems of mathematical psychology , 1997 .

[34]  Lawrence Weiskrantz,et al.  Consciousness and Commentaries , 1998 .

[35]  William M. Petrusic,et al.  Probing the locus of confidence judgments: experiments on the time to determine confidence. , 1998 .

[36]  J V Baranski,et al.  Probing the locus of confidence judgments: experiments on the time to determine confidence. , 1998, Journal of experimental psychology. Human perception and performance.

[37]  T. Van Zandt,et al.  ROC curves and confidence judgements in recognition memory. , 2000, Journal of experimental psychology. Learning, memory, and cognition.

[38]  T. Zandt.  ROC curves and confidence judgments in recognition memory. , 2000 .

[39]  Geoffrey R. Loftus,et al.  Accounts of the confidence-accuracy relation in recognition memory , 2000, Psychonomic bulletin & review.

[40]  H. Pashler,et al.  Confidence and Accuracy of Near-Threshold Discrimination Responses , 2001, Consciousness and Cognition.

[41]  J. Eastwood,et al.  Perception without awareness: perspectives from cognitive psychology , 2001, Cognition.

[42]  Philip A. Higham,et al.  Strong cues are not necessarily weak: Thomson and Tulving (1970) and the encoding specificity principle revisited , 2002, Memory & cognition.