A Test of the Theory of DSS Design for User Calibration: The Effects of Expressiveness and Visibility on User Calibration

This paper reports a test of the theory of decision support systems design for user calibration that compares the efficiency of the visual computing paradigm with that of the conventional text paradigm over varied levels of problem novelty. Perfect user calibration exists when a user’s confidence in a decision equals the quality of the decision. The laboratory study reported here compared the effects on user calibration of problems depicted either using a text paradigm or visual computing paradigm. The results support the theory. When problems are new and novel, visual depiction improves user calibration. As problems became more familiar and problem novelty decreases, no difference was found in user calibration between subjects exposed to visibility diagrams and those exposed to a traditional text paradigm.

[1]  K. Thompson.  Cognitive and Analytical Psychology Howard Gardner .Frames of Mind: The Theory of Multiple Intelligences. New York, Basic Books, 1983. , 1985 .

[2]  T. Helstrup,et al.  One, two, or three memories? A problem-solving approach to memory for performed acts ☆ , 1987 .

[3]  Lawrence D. Phillips,et al.  The ‘true probability’ problem , 1970 .

[4]  Cheri Speier,et al.  The Influence of Query Interface Design on Decision-Making Performance , 2003, MIS Q..

[5]  G. Keren.  Calibration and probability judgements: Conceptual and methodological issues , 1991 .

[6]  B. Schneiderman,et al.  Designing the User Interface. Strategies for Effective Human-Computer Interaction , 1992 .

[7]  Ben Shneiderman,et al.  Readings in information visualization - using vision to think , 1999 .

[8]  A. H. Murphy,et al.  The expected value of frequency calibration , 1990 .

[9]  Herbert A. Simon,et al.  Why a Diagram is (Sometimes) Worth Ten Thousand Words , 1987 .

[10]  A. H. Murphy,et al.  Scoring rules in probability assessment and evaluation , 1970 .

[11]  Herbert A. Simon,et al.  Why a diagram is (sometimes) worth 10, 000 word , 1987 .

[12]  Ben Shneiderman,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction , 1998 .

[13]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[14]  M TODA.  MEASUREMENT OF SUBJECTIVE PROBABILITY DISTRIBUTIONS. TECHN DOCUM REP ESD-TDR-63-407. , 1963, Technical documentary report. United States. Air Force. Systems Command. Electronic Systems Division.

[15]  Robert L. Winkler,et al.  Probabilistic Prediction: Some Experimental Results , 1971 .

[16]  D. F. Marks,et al.  Visual imagery differences in the recall of pictures. , 1973, British journal of psychology.

[17]  Mark S. Silver,et al.  Decision Support Systems: Directed and Nondirected Change , 1990, Inf. Syst. Res..

[18]  George M. Kasper,et al.  A Theory of Decision Support System Design for User Calibration , 1996, Inf. Syst. Res..

[19]  George M. Kasper,et al.  Animation in User Interfaces Designed for Decision Support Systems: The Effects of Image Abstraction, Transition, and Interactivity on Decision Quality* , 1997 .

[20]  P. Johnson-Laird,et al.  PSYCHOLOGICAL SCIENCE Research Article HOW DIAGRAMS CAN IMPROVE REASONING , 2022 .

[21]  Bonnie M. Muir,et al.  Trust Between Humans and Machines, and the Design of Decision Aids , 1987, Int. J. Man Mach. Stud..

[22]  B. Fischhoff,et al.  Calibration of probabilities: the state of the art to 1980 , 1982 .

[23]  H. Gardner,et al.  Frames of Mind: The Theory of Multiple Intelligences , 1983 .

[24]  Robert M. Rippey.  A COMPARISON OF FIVE DIFFERENT SCORING FUNCTIONS FOR CONFIDENCE TESTS1 , 1970 .