Decision Quality, Confidence, and Commitment with Expert Systems: An Experimental Study

This experiment investigates the relationship between results produced by an expert system (ES), its physical structure and features, and the user's commitment and confidence in its solutions. One hundred one subjects among four MPA programs were asked to evaluate ten applicants for a government budget officer position. Among the findings: decision makers who utilized the ES made higher quality decisions but exhibited less confidence and commitment than did those who worked without an ES; decisionmaker involvement in the decision process corrected for this disparity. We discuss the implications for ES design and future research.

[1]  David H. Coursey,et al.  Expert System Technology for Managerial Applications: A Typology , 1989 .

[2]  Norm Parry,et al.  Managing information technology: Transforming county governments in the 1990s: By Patricia T. Fletcher, Stuart I. Bretschneider, Donald A. Marchand, et al. Syracuse, NY: Syracuse University School of Information Studies, 1992. $34.95. 213 pages. ISBN 0-9623807-0-9 , 1993 .

[3]  S. Hadden,et al.  Intelligent advisory systems for managing and disseminating information , 1986 .

[4]  James C. Hershauer.  The New Science of Management Decision , 1978 .

[5]  Izak Benbasat,et al.  An evaluation of empirical research in managerial support systems , 1990, Decis. Support Syst..

[6]  Barry M. Staw,et al.  Knee-deep in the big muddy : the effect of personal responsibility and decision consequences upon commitment to a previously chosen course of action / 220 , 1974 .

[7]  Barry M. Staw,et al.  Commitment to a Policy Decision: A Multi-Theoretical Perspective. , 1978 .

[8]  Andrew P. Sage,et al.  Behavioral and Organizational Considerations in the Design of Information Systems and Processes for Planning and Decision Support , 1981, IEEE Transactions on Systems, Man, and Cybernetics.

[9]  James A. Senn,et al.  Information System Structure and Purchasing Decision Effectiveness , 1974 .

[10]  Barry Bozeman,et al.  Computers and commitment to a public management decision: An experiment , 1989 .

[11]  Izak Benbasat,et al.  AN EXPERIMENTAL EVALUATION OF THE RELATIONSHIP OF UNCERTAINTY IN THE ENVIRONMENT TO INFORMATION USED BY DECISION MAKERS , 1975 .

[12]  Robert C. Joyner,et al.  Computer Augmented Organizational Problem Solving , 1970 .

[13]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[14]  Morten Lind,et al.  Coping with complexity , 2006, Nature.

[15]  Joel B. Cohen,et al.  The social animal. , 1973 .

[16]  Nicholas Rescher,et al.  Rationality: A Philosophical Inquiry into the Nature and the Rationale of Reason , 1988 .

[17]  Daniel J. Power,et al.  AN EMPIRICAL ASSESSMENT OF COMPUTER‐ASSISTED DECISION ANALYSIS* , 1986 .

[18]  Gordon B. Davis,et al.  Management information systems : conceptual foundations, structure, and development , 1985 .

[19]  P. Meehl,et al.  Antecedent probability and the efficiency of psychometric signs, patterns, or cutting scores. , 1955, Psychological bulletin.

[20]  Martin D. Goslar,et al.  Decision support systems: an empirical assessment for decision making , 1986 .

[21]  Gerardine DeSanctis,et al.  ICIS Paper: Methodological Issues in Experimental IS Research: Experiences and Recommendations , 1985, MIS Q..

[22]  David H. Coursey.  Information Credibility and Choosing Policy Alternatives: An Experimental Test of Cognitive-Response Theory , 1992 .

[23]  P. Johnson-Laird.  The computer and the mind: an introduction to cognitive science , 1988 .

[24]  Michael Goul,et al.  Using a Knowledge-Based Decision Support System in Strategic Planning Decisions: An Empirical Study , 1986, J. Manag. Inf. Syst..

[25]  A. Tversky,et al.  Judgment under Uncertainty , 1982 .

[26]  G. Huber.  Managerial decision making , 1980 .

[27]  Barry M. Staw,et al.  Knee-deep in the Big Muddy: A study of escalating commitment to a chosen course of action. , 1976 .

[28]  A. Tversky,et al.  Prospect Theory : An Analysis of Decision under Risk Author ( s ) : , 2007 .

[29]  A. Rosenfeld,et al.  IEEE TRANSACTIONS ON SYSTEMS , MAN , AND CYBERNETICS , 2022 .

[30]  Randall Steeb,et al.  A Computer-Based Interactive System for Group Decisionmaking , 1981, IEEE Transactions on Systems, Man, and Cybernetics.

[31]  Joel Brockner,et al.  Escalation of Commitment to an Ineffective Course of Action: The Effect of Feedback Having Negative Implications for Self-Identity. , 1986 .

[32]  L. Richard Ye,et al.  The Impact of Explanation Facilities in User Acceptance of Expert System Advice , 1995, MIS Q..

[33]  Bo K. Wong,et al.  Expert system applications in business: A review and analysis of the literature (1977-1993) , 1995, Inf. Manag..

[34]  Gerardine DeSanctis,et al.  Computer-Based Support for Group Problem-Finding: An Experimental Investigation , 1988, MIS Q..

[35]  B. Bozeman,et al.  Scientific and Technical Information in Public Management , 1982 .

[36]  George P. Huber,et al.  DECISION SUPPORT SYSTEMS FOR ILL‐STRUCTURED PROBLEMS: AN EMPIRICAL STUDY , 1987 .

[37]  L. Festinger,et al.  A Theory of Cognitive Dissonance , 2017 .

[38]  B. Bozeman,et al.  Truth and Credibility in Sincere Policy Analysis , 1989 .

[39]  Izak Benbasat,et al.  Process Tracing Methods in Decision Support Systems Research: Exploring the Black Box , 1987, MIS Q..

[40]  Alex Goodall,et al.  The guide to expert systems , 1985 .

[41]  T. Allen.  Managing the flow of technology , 1977 .

[42]  Starr Roxanne Hiltz,et al.  Computer Support for Group Versus Individual Decisions , 1982, IEEE Trans. Commun..

[43]  Daniel J. Power,et al.  Impacts of problem structure and computerized decision aids on decision attitudes and behaviors , 1994, Inf. Manag..

[44]  Ramesh Sharda,et al.  Decision support system effectiveness: a review and an empirical test , 1988 .

[45]  Bruce Cheek,et al.  The Use of Different Information Channels in an Organizational Context , 1994 .

[46]  Barry M. Staw,et al.  Escalation: The Determinants of Commitment to a Chosen Course of Action , 1977 .

[47]  E. Aronson,et al.  The social animal. , 1996 .

[48]  Charles H. Schwenk.  Information, Cognitive Biases, and Commitment to a Course of Action , 1986 .

[49]  Donald A. Marchand,et al.  Information Management in Counties@@@Managing Information Technology: Transforming County Governments in the 1990s , 1994 .

[50]  Michael G Samet,et al.  Empirical Evaluation of a Decision-Analytic Aid1 , 1980 .

[51]  S. Oskamp.  OVERCONFIDENCE IN CASE-STUDY JUDGMENTS. , 1965, Journal of consulting psychology.

[52]  J. Weiss.  Coping with complexity: An experimental study of public policy decision-making , 1982 .

[53]  B. Bozeman,et al.  Credibility Logic and Policy Analysis , 1987 .

[54]  Barry Bozeman,et al.  Public management information systems: theory and prescription , 1986 .

[55]  Tor Guimaraes,et al.  Exploring the Factors Associated with Expert Systems Success , 1995, MIS Q..

[56]  Barry Bozeman,et al.  THE CREDIBILITY OF POLICY ANALYSIS: BETWEEN METHOD AND USE , 1986 .

[57]  Conrad Bock,et al.  Expert System Applications , 2012, Symbolic Computation.

[58]  Edward J. Conlon,et al.  The moderating effects of strategy, visibility, and involvement on allocation behavior: An extension of Staw's escalation paradigm. , 1980 .

[59]  J. Bernard Keys,et al.  Managerial Decision Making:Japan vs. the US , 1985 .

[60]  Patrick Henry Winston,et al.  Intelligent Advisory Systems , 1986 .

[61]  Herbert A. Simon,et al.  The new science of management decision , 1960 .

[62]  Charles A. O'Reilly,et al.  Variations in Decision Makers' Use of Information Sources: The Impact of Quality and Accessibility of Information , 1982 .