Two-stage dynamic signal detection: a theory of choice, decision time, and confidence.

The 3 most often-used performance measures in the cognitive and decision sciences are choice, response or decision time, and confidence. We develop a random walk/diffusion theory-2-stage dynamic signal detection (2DSD) theory-that accounts for all 3 measures using a common underlying process. The model uses a drift diffusion process to account for choice and decision time. To estimate confidence, we assume that evidence continues to accumulate after the choice. Judges then interrupt the process to categorize the accumulated evidence into a confidence rating. The model explains all known interrelationships between the 3 indices of performance. Furthermore, the model also accounts for the distributions of each variable in both a perceptual and general knowledge task. The dynamic nature of the model also reveals the moderating effects of time pressure on the accuracy of choice and confidence. Finally, the model specifies the optimal solution for giving the fastest choice and confidence rating for a given level of choice and confidence accuracy. Judges are found to act in a manner consistent with the optimal solution when making confidence judgments.

[1]  J. Jastrow,et al.  On small differences in sensation , 1884 .

[2]  V. Henmon,et al.  The relation of the time of a judgment to its accuracy. , 1911 .

[3]  Donald McEwen Johnson,et al.  Confidence and speed in the two-category judgment , 1939 .

[4]  Leon Festinger,et al.  Studies in decision: I. Decision-time, relative frequency of judgment and subjective confidence as related to physical stimulus difference. , 1943 .

[5]  G. A. Barnard,et al.  Sequential Tests in Industrial Statistics , 1946 .

[6]  J. Wolfowitz,et al.  Optimum Character of the Sequential Probability Ratio Test , 1948 .

[7]  Feller William,et al.  An Introduction To Probability Theory And Its Applications , 1950 .

[8]  G. Brier.  VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY , 1950 .

[9]  F. W. Irwin,et al.  Tests of two theories of decision in an expanded judgment situation. , 1956, Journal of experimental psychology.

[10]  J K ADAMS,et al.  A confidence scale defined in terms of expected percentages. , 1957, The American journal of psychology.

[11]  James P. Egan,et al.  Recognition memory and the operating characteristic. , 1958 .

[12]  M. Stone.  Models for choice-reaction time , 1960 .

[13]  R J AUDLEY,et al.  A stochastic model for individual choice behavior. , 1960, Psychological review.

[14]  P. A. Adams,et al.  Realism of confidence judgments. , 1961, Psychological review.

[15]  R PIERREL,et al.  Some relationships between comparative judgment, confidence, and decision-time in weight-lifting. , 1963, The American journal of psychology.

[16]  Thornton Page,et al.  The Scientist Speculates: An Anthology of Partly-baked Ideas , 1964 .

[17]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[18]  W. Edwards.  Optimal strategies for seeking information: Models for statistics, choice reaction times, and human information processing ☆ , 1965 .

[19]  R. Audley,et al.  SOME ALTERNATIVE STOCHASTIC MODELS OF CHOICE1 , 1965 .

[20]  William K. Estes,et al.  Reaction time in relation to display size and correctness of response in forced-choice visual signal detection , 1966 .

[21]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[22]  J. F. Schouten,et al.  Reaction time and accuracy. , 1967, Acta psychologica.

[23]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1967 .

[24]  J. Pfanzagl,et al.  Remarks on the measurement of subjective probability and information , 1967 .

[25]  David R. Cox,et al.  The Theory of Stochastic Processes , 1967, The Mathematical Gazette.

[26]  Donald Laming,et al.  Information theory of choice-reaction times , 1968 .

[27]  A. H. Murphy,et al.  “Good” Probability Assessors , 1968 .

[28]  J. Andel.  Sequential Analysis , 2022, The SAGE Encyclopedia of Research Design.

[29]  Carl-Axel S. Staël von Holstein,et al.  Measurement of subjective probability , 1970 .

[30]  W. Edwards,et al.  Response strategies in a two-choice reaction task with a continuous cost for time , 1971 .

[31]  Richard G. Swensson,et al.  The elusive tradeoff: Speed vs accuracy in visual discrimination tasks , 1972 .

[32]  Gregory K. Miller,et al.  Elements of Applied Stochastic Processes , 1972 .

[33]  R. Pike.  Response latency models for signal detection. , 1973, Psychological review.

[34]  A. H. Murphy.  A New Vector Partition of the Probability Score , 1973 .

[35]  A Model for Confidence Judgments in Choice Tasks , 1974 .

[36]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[37]  M. S. Mayzner,et al.  Human information processing : tutorials in performance and cognition , 1975 .

[38]  S. Link,et al.  A sequential theory of psychological discrimination , 1975 .

[39]  A. H. Murphy,et al.  Reliability of Subjective Probability Forecasts of Precipitation and Temperature , 1977 .

[40]  Wayne A. Wickelgren,et al.  Speed-accuracy tradeoff and information processing dynamics , 1977 .

[41]  Roger Ratcliff,et al.  A Theory of Memory Retrieval. , 1978 .

[42]  D. Vickers.  Decision processes in visual perception , 1979 .

[43]  L. A. Goodman,et al.  Measures of association for cross classifications , 1979 .

[44]  William R. Ferrell,et al.  A model of calibration for subjective probabilities , 1980 .

[45]  B. Fischhoff,et al.  Reasons for confidence. , 1980 .

[46]  B. Fischhoff,et al.  Journal of Experimental Psychology: Human Learning and Memory , 1980 .

[47]  D Vickers,et al.  Effects of alternating set for speed or accuracy on response time, accuracy and confidence in a unidimensional discrimination task. , 1982, Acta psychologica.

[48]  David S. Cordray,et al.  Similarity and choice : papers in honour of Clyde Coombs , 1982 .

[49]  B. Fischhoff,et al.  Calibration of probabilities: the state of the art to 1980 , 1982 .

[50]  James T. Townsend,et al.  The Stochastic Modeling of Elementary Psychological Processes , 1983 .

[51]  R A Heath,et al.  Random-Walk and Accumulator Models of Psychophysical Discrimination: A Critical Evaluation , 1984, Perception.

[52]  T. O. Nelson,et al.  A comparison of current measures of the accuracy of feeling-of-knowing predictions. , 1984, Psychological bulletin.

[53]  Shawn P. Curley,et al.  Conditional distribution analyses of probabilistic forecasts , 1985 .

[54]  D. Vickers,et al.  Accumulator and Random-Walk Models of Psychophysical Discrimination: A Counter-Evaluation , 1985, Perception.

[55]  Philip L. Smith,et al.  Experimental paradigms emphasising state or process limitations: II effects on confidence , 1985 .

[56]  Philip L. Smith,et al.  Experimental paradigms emphasising state or process limitations: I effects on speed-accuracy tradeoffs , 1985 .

[57]  R. Duncan Luce,et al.  Response Times: Their Role in Inferring Elementary Mental Organization , 1986 .

[58]  Reinhard Suck,et al.  Progress in mathematical psychology , 1987 .

[59]  G. Keren,et al.  On the ability of monitoring non-veridical perceptions and uncertain knowledge: some calibration studies. , 1988, Acta psychologica.

[60]  John R. Anderson.  The Adaptive Character of Thought , 1990 .

[61]  T. O. Nelson.  Metamemory: A Theoretical Framework and New Findings , 1990 .

[62]  Philip L. Smith.  A note on the distribution of response times for a random walk Gaussian increments , 1990 .

[63]  Neil A. Macmillan,et al.  Detection Theory: A User's Guide , 1991 .

[64]  G. Gigerenzer,et al.  Probabilistic mental models: a Brunswikian theory of confidence. , 1991, Psychological review.

[65]  Jerome R. Busemeyer,et al.  Linking together different measures of preference: A dynamic model of matching derived from decision field theory , 1992 .

[66]  A. Tversky,et al.  The weighing of evidence and the determinants of confidence , 1992, Cognitive Psychology.

[67]  J. Townsend,et al.  Fundamental derivations from decision field theory , 1992 .

[68]  R Ratcliff,et al.  Testing global memory models using ROC curves. , 1992, Psychological review.

[69]  N. Sanders,et al.  Journal of behavioral decision making: "The need for contextual and technical knowledge in judgmental forecasting", 5 (1992) 39-52 , 1992 .

[70]  Anders Winman,et al.  Calibration of sensory and cognitive judgments : Two different accounts , 1993 .

[71]  Rami Zwick,et al.  Comparing the calibration and coherence of numerical and verbal probability judgments , 1993 .

[72]  J. Townsend,et al.  Decision field theory: a dynamic-cognitive approach to decision making in an uncertain environment. , 1993, Psychological review.

[73]  P. Juslin,et al.  Realism of confidence in sensory discrimination: The underconfidence phenomenon , 1993, Perception & psychophysics.

[74]  Ola Svenson,et al.  Time pressure and stress in human judgment and decision making , 1993 .

[75]  P. Juslin.  The Overconfidence Phenomenon as a Consequence of Informal Experimenter-Guided Selection of Almanac Items , 1994 .

[76]  I. Erev,et al.  Simultaneous Over- and Underconfidence: The Role of Error in Judgment Processes. , 1994 .

[77]  L. Thurstone.  A law of comparative judgment. , 1994 .

[78]  Claudia González-Vallejo,et al.  Statement Verification: A Stochastic Model of Judgment and Response. , 1994 .

[79]  William R. Shadish,et al.  Combining estimates of effect size. , 1994 .

[80]  B. Mellers,et al.  Similarity and Choice. , 1994 .

[81]  Ab McClelland,et al.  The calibration of subjective probabilities: Theories and models 1980-1993 , 1994 .

[82]  A. Yonelinas.  Receiver-operating characteristics in recognition memory: evidence for a dual-process model. , 1994, Journal of experimental psychology. Learning, memory, and cognition.

[83]  J. Baranski,et al.  The calibration and resolution of confidence in perceptual judgments , 1994, Perception & psychophysics.

[84]  A. Tversky,et al.  Support theory: A nonextensional representation of subjective probability. , 1994 .

[85]  Philip L. Smith.  Psychophysically principled models of visual simple reaction time. , 1995 .

[86]  L. Hedges,et al.  The Handbook of Research Synthesis , 1995 .

[87]  D. Budescu,et al.  Processing Linguistic Probabilities: General Principles and Empirical Evidence , 1995 .

[88]  P. Juslin,et al.  Can overconfidence be used as an indicator of reconstructive rather than retrieval processes? , 1995, Cognition.

[89]  J. Baranski,et al.  On the calibration of knowledge and perception. , 1995, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[90]  J. Schall,et al.  Neural Control of Voluntary Movement Initiation , 1996, Science.

[91]  T. O. Nelson.  Consciousness and metacognition. , 1996 .

[92]  Thomas S. Wallsten,et al.  An Analysis of Judgment Research Analyses , 1996 .

[93]  W. Ferrell,et al.  The Hard-Easy Effect in Subjective Probability Calibration , 1996 .

[94]  T. O. Nelson,et al.  Measuring ordinal association in situations that contain tied scores. , 1996, Psychological bulletin.

[95]  Ido Erev,et al.  On the Importance of Random Error in the Study of Probability Judgment. Part I: New Theoretical Developments , 1997 .

[96]  David V. Budescu,et al.  On the Importance of Random Error in the Study of Probability Judgment. Part II: Applying the Stochastic Judgment Model to Detect Systematic Trends , 1997 .

[97]  A. Koriat.  Monitoring one's own knowledge during study : A cue-utilization approach to judgments of learning , 1997 .

[98]  The meta-level versus object-level distinction (and other issues) in formulations of metacognition. , 1997 .

[99]  A. Tversky,et al.  Heuristics and Biases: Unpacking, Repacking, and Anchoring: Advances in Support Theory , 2002 .

[100]  Diederich,et al.  Dynamic Stochastic Models for Decision Making under Time Constraints , 1997, Journal of mathematical psychology.

[101]  A. Diederich,et al.  Evaluating and Combining Subjective Probability Estimates , 1997 .

[102]  P. Juslin,et al.  Thurstonian and Brunswikian origins of uncertainty in judgment: a sampling model of confidence in sensory discrimination. , 1997, Psychological review.

[103]  R. Nosofsky,et al.  An exemplar-based random walk model of speeded classification. , 1997, Psychological review.

[104]  J V Baranski,et al.  Probing the locus of confidence judgments: experiments on the time to determine confidence. , 1998, Journal of experimental psychology. Human perception and performance.

[105]  Jeffrey N. Rouder,et al.  Modeling Response Times for Two-Choice Decisions , 1998 .

[106]  T. Wallsten,et al.  Dissociating judgment from response processes in statement verification: the effects of experience on each component. , 1999, Journal of experimental psychology. Learning, memory, and cognition.

[107]  M. Shadlen,et al.  Neural correlates of a decision in the dorsolateral prefrontal cortex of the macaque , 1999, Nature Neuroscience.

[108]  R. Ratcliff,et al.  Connectionist and diffusion models of reaction time. , 1999, Psychological review.

[109]  Jeffrey N. Rouder,et al.  Assessing the roles of change discrimination and luminance integration: evidence for a hybrid race model of perceptual decision making in luminance discrimination. , 2000, Journal of experimental psychology. Human perception and performance.

[110]  Philip L. Smith,et al.  Stochastic Dynamic Models of Response Time and Accuracy: A Foundational Primer. , 2000, Journal of mathematical psychology.

[111]  P. Juslin,et al.  Naive empiricism and dogmatism in confidence research: a critical examination of the hard-easy effect. , 2000, Psychological review.

[112]  H Gu,et al.  The effects of averaging subjective probability estimates between and within judges. , 2000, Journal of experimental psychology. Applied.

[113]  T. Zandt,et al.  A comparison of two response time models applied to perceptual matching , 2000, Psychonomic bulletin & review.

[114]  Trisha Van Zandt,et al.  Roc curves and confidence judgments in recognition memory , 2000 .

[115]  Jeffrey N. Rouder,et al.  A diffusion model account of masking in two-choice letter identification. , 2000, Journal of experimental psychology. Human perception and performance.

[116]  Geoffrey R. Loftus,et al.  Accounts of the confidence-accuracy relation in recognition memory , 2000, Psychonomic bulletin & review.

[117]  Ashby,et al.  A Stochastic Version of General Recognition Theory. , 2000, Journal of mathematical psychology.

[118]  T. Zandt,et al.  How to fit a response time distribution , 2000, Psychonomic bulletin & review.

[119]  James L. McClelland,et al.  The time course of perceptual choice: the leaky, competing accumulator model. , 2001, Psychological review.

[120]  Hal R. Arkes,et al.  Overconfidence in Judgmental Forecasting , 2001 .

[121]  W. Newsome,et al.  Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. , 2001, Journal of neurophysiology.

[122]  J. Scott Armstrong,et al.  Principles of forecasting : a handbook for researchers and practitioners , 2001 .

[123]  R. Ratcliff,et al.  Multialternative decision field theory: a dynamic connectionist model of decision making. , 2001, Psychological review.

[124]  J. Gold,et al.  Neural computations that underlie decisions about sensory stimuli , 2001, Trends in Cognitive Sciences.

[125]  M. Dougherty,et al.  Integration of the ecological and error models of overconfidence using a multiple-trace memory model. , 2001, Journal of experimental psychology. General.

[126]  D. Vickers.  WHERE DOES THE BALANCE OF EVIDENCE LIE WITH RESPECT TO CONFIDENCE ? , 2001 .

[127]  R. Ratcliff.  A diffusion model account of response time and accuracy in a brightness discrimination task: Fitting real data and failing to fit fake but plausible data , 2002, Psychonomic bulletin & review.

[128]  J. Gold,et al.  Banburismus and the Brain Decoding the Relationship between Sensory Stimuli, Decisions, and Reward , 2002, Neuron.

[129]  Derek J. Koehler,et al.  Heuristics and Biases: The Calibration of Expert Judgment: Heuristics and Biases Beyond the Laboratory , 2002 .

[130]  Xiao-Jing Wang,et al.  Probabilistic Decision Making by Slow Reverberation in Cortical Circuits , 2002, Neuron.

[131]  D. Kahneman,et al.  Heuristics and Biases: The Psychology of Intuitive Judgment , 2002 .

[132]  Gerd Gigerenzer,et al.  Models of ecological rationality: the recognition heuristic. , 2002, Psychological review.

[133]  R. Romo,et al.  Neuronal correlates of decision-making in secondary somatosensory cortex , 2002, Nature Neuroscience.

[134]  Scott D. Brown,et al.  Quantile maximum likelihood estimation of response time distributions , 2002, Psychonomic bulletin & review.

[135]  R. Ratcliff,et al.  Estimating parameters of the diffusion model: Approaches to dealing with contaminant reaction times and parameter variability , 2002, Psychonomic bulletin & review.

[136]  Errol C. Caby.  Elements of Applied Stochastic Processes , 2003, Technometrics.

[137]  William M Petrusic,et al.  Judging confidence influences decision processing in comparative judgments , 2003, Psychonomic bulletin & review.

[138]  Adele Diederich,et al.  Simple matrix methods for analyzing diffusion models of choice probability, choice response time, and simple response time , 2003 .

[139]  M. Lee,et al.  Evidence accumulation in decision making: Unifying the “take the best” and the “rational” models , 2004, Psychonomic bulletin & review.

[140]  Jeffrey N Rouder,et al.  A comment on Heathcote, Brown, and Mewhort’s QMLE method for response time distributions , 2004, Psychonomic bulletin & review.

[141]  Mildred M Maldonado-Molina,et al.  Response reversals in recognition memory. , 2004, Journal of experimental psychology. Learning, memory, and cognition.

[142]  James L. McClelland,et al.  Loss aversion and inhibition in dynamical models of multialternative choice. , 2004, Psychological review.

[143]  Philip L. Smith,et al.  A comparison of sequential sampling models for two-choice reaction time. , 2004, Psychological review.

[144]  Joseph G. Johnson,et al.  A dynamic, stochastic, computational model of preference reversal phenomena. , 2005, Psychological review.

[145]  James T. Townsend,et al.  Building bridges between neural models and complex decision making behaviour , 2006, Neural Networks.

[146]  Jonathan D. Cohen,et al.  The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. , 2006, Psychological review.

[147]  Edgar C Merkle,et al.  An application of the poisson race model to confidence calibration. , 2006, Journal of experimental psychology. General.

[148]  R. Ratcliff,et al.  A model of the go/no-go task. , 2007, Journal of experimental psychology. General.

[149]  J. Gold,et al.  The neural basis of decision making. , 2007, Annual review of neuroscience.

[150]  Eric-Jan Wagenmakers,et al.  An EZ-diffusion model for response time and accuracy , 2007, Psychonomic bulletin & review.

[151]  Winston R. Sieck,et al.  Option fixation: A cognitive contributor to overconfidence , 2007 .

[152]  R. Clark,et al.  Recognition memory and the medial temporal lobe: a new perspective , 2007, Nature Reviews Neuroscience.

[153]  Timothy D. Hanks,et al.  Neurobiology of decision making: An intentional framework , 2008 .

[154]  Lyle Brenner,et al.  Perspectives on Probability Judgment Calibration , 2008 .

[155]  Andreas Voss,et al.  A fast numerical algorithm for the estimation of diffusion model parameters , 2008 .

[156]  W. Singer,et al.  Better than conscious? : decision making, the human mind, and implications for institutions , 2008 .

[157]  Scott D. Brown,et al.  The simplest complete model of choice response time: Linear ballistic accumulation , 2008, Cognitive Psychology.

[158]  R. Ratcliff,et al.  Modeling confidence and response time in recognition memory. , 2009, Psychological review.

[159]  D. Wolpert,et al.  Changing your mind: a computational mechanism of vacillation , 2009, Nature.

[160]  M. Shadlen,et al.  Representation of Confidence Associated with a Decision by Neurons in the Parietal Cortex , 2009, Science.

[161]  Timothy J. Pleskac,et al.  Random error in judgment: The contribution of encoding and retrieval processes , 2009 .

[162]  B. Fischhoff,et al.  Judgment and decision making. , 2012, Wiley interdisciplinary reviews. Cognitive science.

[163]  R. Woodworth.  Archives of psychology , 2010 .

[164]  Rubén Moreno-Bote,et al.  Decision Confidence and Uncertainty in Diffusion Models with Partially Correlated Neuronal Integrators , 2010, Neural Computation.

[165]  R. Bogacz,et al.  The neural basis of the speed–accuracy tradeoff , 2010, Trends in Neurosciences.

[166]  Robert G Pachella,et al.  The Interpretation of Reaction Time in Information-Processing Research 1 , 1973, Human Information Processing.