Calibration curves, scatterplots and the distinction between general knowledge and perceptual tasks

Abstract Calibration curves and other methods of description and analysis (such as scatterplots and correlations) which derive from the individual differences tradition in psychology are examined in this article. Three tasks covering a range of cognitive ability were employed: the Raven's Progressive Matrices (RPM) test, a multiple choice synonyms Vocabulary test, and a perceptual test of Line Length. Scatterplots of accuracy and confidence ratings reveal that (a) the overconfidence that characterizes Vocabulary tasks is due to a small number of items containing a “familiar attractor” alternative answer; (b) the RPM shows good calibration; and (c) the underconfidence phenomena in the perceptual task is due to a tendency to ascribe essentially the same confidence rating to a number of items that differ in difficulty. The results of correlational analyses support the view that there exists a trait of self-confidence that is separate from traits based on accuracy measures. Overall, males show a somewhat higher degree of self-confidence. Finally, practice has a negligible effect on confidence ratings.

[1]  L. Stankov,et al.  Self-confidence and performance on tests of cognitive abilities , 1997 .

[2]  Ruth B. Ekstrom,et al.  MANUAL FOR KIT OF REFERENCE TESTS FOR COGNITIVE FACTORS (REVISED 1963) , 1963 .

[3]  D. Feltz.  Gender Differences in the Causal Elements of Self-Efficacy on a High Avoidance Motor Task , 1988 .

[4]  G. Gigerenzer,et al.  Probabilistic mental models: a Brunswikian theory of confidence. , 1991, Psychological review.

[5]  G. Keren.  Calibration and probability judgements: Conceptual and methodological issues , 1991 .

[6]  L. Stankov,et al.  Confidence judgments in studies of individual differences , 1996 .

[7]  D. Vickers.  Decision processes in visual perception , 1979 .

[8]  Gregory Schraw,et al.  Does a General Monitoring Skill Exist , 1995 .

[9]  P. Juslin.  The Overconfidence Phenomenon as a Consequence of Informal Experimenter-Guided Selection of Almanac Items , 1994 .

[10]  P. Juslin,et al.  Thurstonian and Brunswikian origins of uncertainty in judgment: a sampling model of confidence in sensory discrimination. , 1997, Psychological review.

[11]  J. Carroll.  Human Cognitive Abilities-a sur-vey of factor-analytic studies , 1993 .

[12]  S. Lichtenstein,et al.  Do those who know more also know more about how much they know?*1 , 1977 .

[13]  P. Juslin,et al.  Realism of confidence in sensory discrimination: The underconfidence phenomenon , 1993, Perception & psychophysics.

[14]  J. Baranski,et al.  The calibration and resolution of confidence in perceptual judgments , 1994, Perception & psychophysics.

[15]  Lazar Stankov,et al.  Age differences in the realism of confidence judgements: A calibration study using tests of fluid and crystallized intelligence , 1996 .

[16]  M. Björkman.  Internal Cue Theory: Calibration and Resolution of Confidence in General Knowledge , 1994 .

[17]  Raymond B. Cattell,et al.  Handbook of multivariate experimental psychology , 1968 .

[18]  John D. Van Horn,et al.  Thinking about Human Abilities , 1988 .