Utilization-Oriented Evaluation of Decision Support Systems

A major purpose in the design of evaluations is to provide information. This information is useful to the extent that it can and will be used to shape decisions. Thus one way to ensure that evaluation information will be used is to make the information relevant to decisions that are to be made. USE, a model for the systematic evaluation of decision support systems (DSS's), is described. USE focuses on the utilization of evaluation information as a primary objective in evaluation design. Utilization is optimized in the model by 1) directing the evaluation toward decisions that are to be made with regard to the system, 2) ensuring that the evaluation information is timely, 3) incorporating a designer-user-evaluator team, and 4) using an evaluation template to enable rapid evaluation of potential aids. Procedures for choosing measures of effectiveness and the types of evaluation that can be conducted are also discussed.

[1]  Paul E. Lehner,et al.  User and R&D specialist evaluation of decision-support systems , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[2]  Edgar H. Sibley.  How to Select and Evaluate a DBMS , 1985 .

[3]  Russell L. Ackoff,et al.  Management misinformation systems , 1967 .

[4]  L. Phillips.  A theory of requisite decision models , 1984 .

[5]  Jacob Marschak,et al.  ECONOMICS OF INFORMATION SYSTEMS , 1971 .

[6]  Andrew P. Sage.  A methodological framework for systemic design and evaluation of computer aids for planning and decision support , 1981 .

[7]  Michael E. Atwood,et al.  Paper Simulation Techniques in User Requirements Analysis for Interactive Computer Systems1 , 1979 .

[8]  Sandra H. Rouse,et al.  Human information seeking and design of information systems , 1984, Inf. Process. Manag..

[9]  Clayton Lewis,et al.  Designing for usability—key principles and what designers think , 1983, CHI '83.

[10]  W. Edwards.  How to Use Multi-Attribute Utility Measurement for Social Decision Making , 1976 .

[11]  Ira W Cotton.  Cost-benefit analysis of computer graphics systems , 1974 .

[12]  Stephen J. Andriole,et al.  The Design of Microcomputer-Based Personal Decision-Aiding Systems , 1982, IEEE Transactions on Systems, Man, and Cybernetics.

[13]  Ward Edwards,et al.  How to Use Multiattribute Utility Measurement for Social Decisionmaking , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[14]  Ronald A. Howard,et al.  Information Value Theory , 1966, IEEE Trans. Syst. Sci. Cybern..

[15]  Henry C. Lucas,et al.  The use of an interactive information storage and retrieval system in medical research , 1978, CACM.