Exploring the Factors Associated with Expert Systems Success

As the widespread use and company dependency on expert systems (ES) incease, so does the need to assess their value and to ensure implementation success. This study identifies and empirically tests eight major variables proposed in the literature as determinants of ES success, in this case measured in terms of user satisfactions. IBM's Corporate Manufacturing Expert Systems Project Center collected information from 69 project managers to support the study. The results clearly support the hypothesized relationships and suggest the need for ES project managers to pay special attention to these determinants of ES implementation success. ES success is directily related to the quality of developers and the ES shells used, end-user characteristcs, and degree of user involvement in ES development, as each has been defined in this study. For exploratory purposes, the component items for each of these major variables were correlated with the components of user satisfaction. Based on the results, several recommendations are proposed for ES project managers to enhance the likelihood of project success, including: adding problem difficulty as a criteriaon for ES application selection; increasing ES developer training to improve people skills; having the ability to model and use a systems approach in solving business problems; sharping end-user attitudes and expectations regarding ES; improving the selction of domain experts; more thoroughly understanding the ES impact on end-user jobs; restricting the acquistion of ES shells based on a rpopsoed set of criteria; and ensuring a proper match of ES development techniques and tools to the business problem at hand.

[1]  Jay Liebowitz Expert systems for business applications , 1987, Appl. Artif. Intell..

[2]  James C. Wetherbe,et al.  Service Support Levels: An Organizational Approach to End-User Computing , 1986, MIS Q..

[3]  Louis Raymond,et al.  Organizational Characteristics and MIS Success in the Context of Small Business , 1985, MIS Q..

[4]  J. Daniel Couger,et al.  Information systems curriculum recommendations for the 80s: undergraduate and graduate programs , 1982, CACM.

[5]  Henri Barki,et al.  Rethinking the Concept of User Involvement , 1989, MIS Q..

[6]  Paul S. Goodman,et al.  The Human Side of Robotics: How Worker's React to a Robot , 1986 .

[7]  Norman L. Eckel THE IMPACT OF PROBABILISTIC INFORMATION ON DECISION BEHAVIOR AND PERFORMANCE IN AN EXPERIMENTAL GAME , 1983 .

[8]  Tor Guimaraes,et al.  A Guide to Selecting Expert Systems Applications , 1989 .

[9]  Victoria Y. Yoon,et al.  Selection of a good expert system shell for instructional purposes in business , 1992, Inf. Manag..

[10]  Henry C. Lucas,et al.  Why information systems fail , 1975 .

[11]  Jay Liebowitz,et al.  Expert systems for business and management , 1990 .

[12]  Sammy W. Pearson,et al.  Development of a Tool for Measuring and Analyzing Computer User Satisfaction , 1983 .

[13]  Efraim Turban,et al.  Integrating Expert Systems and Decision Support Systems , 1986, MIS Q..

[14]  Ramesh Sharda,et al.  Decision support system effectiveness: a review and an empirical test , 1988 .

[15]  Paul H. Cheney,et al.  Training End Users: An Exploratory Study , 1987, MIS Q..

[16]  David S. Prerau,et al.  Developing and managing expert systems , 1989 .

[17]  Tor Guimaraes,et al.  Selecting expert system development techniques , 1993, Inf. Manag..

[18]  D C McLeod,et al.  The experts. , 1975, Drug intelligence & clinical pharmacy.

[19]  Tor Guimaraes,et al.  Measuring top management satisfaction with the MIS department , 1988 .

[20]  Dan R. Pieptea,et al.  Price and Value of Decision Support Systems , 1987, MIS Q..

[21]  Richard G. Vedder PC‐based expert system shells: some desirable and less desirable characteristics , 1989 .

[22]  Jeffrey E. Kottemann,et al.  A Study of the Relationship Between Decision Model Naturalness and Performance , 1989, MIS Q..

[23]  Dorothy Leonard-Barton,et al.  The case for integrative innovation: an expert system at Digital , 1993 .

[24]  Ephraim R. McLean,et al.  Information Systems Success: The Quest for the Independent Variables , 1992, J. Manag. Inf. Syst..

[25]  Paul Harmon,et al.  Expert systems: tools and applications , 1988 .

[26]  Vice President,et al.  An Introduction to Expert Systems , 1989 .

[27]  W. Doll,et al.  A discrepancy model of end-user computing involvement , 1989 .

[28]  Elias M. Awad,et al.  The systems analyst as a knowledge engineer: can the transition be successfully made? , 1990, SIGBDP '90.

[29]  Donald Smith,et al.  Implementing real world expert systems , 1988 .

[30]  Fred D. Davis Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[31]  Thomas L. Case,et al.  Implementing Expert Systems in R&D , 1991 .

[32]  Terry Anthony Byrd,et al.  Implementation and Use of Expert Systems in Organizations: Perceptions of Knowledge Engineers , 1992, J. Manag. Inf. Syst..

[33]  Larry Kerschberg,et al.  Developing knowledge-based systems: reorganizing the system development life cycle , 1989, CACM.

[34]  David S. Prerau Developing and managing expert systems - proven techniques for business and industry , 1989 .

[35]  George P. Huber,et al.  DECISION SUPPORT SYSTEMS FOR ILL‐STRUCTURED PROBLEMS: AN EMPIRICAL STUDY , 1987 .

[36]  A. V. D. Ven,et al.  Measuring And Assessing Organizations , 1980 .

[37]  David Shpilberg,et al.  ExperTAX sm : an expert system for corporate tax planing , 1986 .

[38]  B. Brabander,et al.  Successful Information System Development in Relation to Situational Factors Which Affect Effective Communication Between MIS-Users and EDP-Specialists , 1984 .

[39]  Thomas J. Beckman,et al.  Selecting expert-system applications , 1991 .

[40]  Peter P. Mykytyn,et al.  End user perceptions of DSS training and DSS usage , 1988 .

[41]  Stephen B. Sloane,et al.  The Use of Artificial Intelligence by the United States Navy: Case Study of a Failure , 1991, AI Mag..

[42]  Denis M. S. Lee Usage Patterns and Sources of Assistance for Personal Computer Users , 1986, MIS Q..

[43]  Richard G. Vedder Five PC-based Expert Systems for Business Reference: An Evaluation , 1989 .

[44]  Michael J. Ginzberg,et al.  Early Diagnosis of MIS Implementation Failure: Promising Results and Unanswered Questions , 1981 .

[45]  James F. Courtney,et al.  A Field Study of Organizational Factors Influencing DSS Success , 1985, MIS Q..

[46]  Paul H. Cheney,et al.  Organizational Factors Affecting the Success of End-User Computing , 1986, J. Manag. Inf. Syst..

[47]  Daniel Robey,et al.  User Attitudes and Management Information System Use , 1979 .

[48]  John J. Sviokla An examination of the impact of expert systems on the firm: the case of XCON , 1990 .

[49]  Gary I. Green,et al.  Perceived Importance of Systems Analysts' Job Skills, Roles, and Non-Salary Incentives , 1989, MIS Q..

[50]  Ann-Marie K. Baronas,et al.  Restoring a Sense of Control During Implementation: How User Involvement Leads to System Acceptance , 1988, MIS Q..

[51]  R. Duda,et al.  Expert Systems Research. , 1983, Science.

[52]  J. Keyes,et al.  Why expert systems fail , 1989 .

[53]  Janet A. Sniezek,et al.  Defining Decision Support Systems: An Empirical Assessment Of End-User Satisfaction , 1989 .

[54]  Henry C. Lucas,et al.  Empirical Evidence for a Descriptive Model of Implementation , 1978, MIS Q..

[55]  J. Casey,et al.  Picking the right expert system application , 1989 .

[56]  Ignizio Introduction to expert systems , 1985 .

[57]  R. V. Schuwer,et al.  Developing and managing expert systems , 1993 .

[58]  Paul S. Goodman,et al.  The Organizational Implications of Robotics , 1985 .

[59]  Arthur G. Money,et al.  The Quantification of Decision Support Benefits Within the Context of Value Analysis , 1988, MIS Q..

[60]  D. Leonard-Barton,et al.  Managerial influence in the implementation of new technology , 1988 .

[61]  Blake Ives,et al.  The measurement of user information satisfaction , 1983, CACM.

[62]  Tor Guimaraes,et al.  The Determinants of DSS Success: An Integrated Model* , 1992 .

[63]  Efraim Turban,et al.  Decision support and expert systems , 1993 .

[64]  James R. Slagle,et al.  A Method for Evaluating Candidate Expert System Applications , 1988, AI Mag..

[65]  Paul H. Cheney,et al.  Concepts, Theory, and Techniques: FACTORS AFFECTING THE PERCEIVED UTILIZATION OF COMPUTER-BASED DECISION SUPPORT SYSTEMS IN THE OIL INDUSTRY , 1982 .

[66]  Edward G. Carmines,et al.  Reliability and Validity Assessment , 1979 .

[67]  Tor Guimaraes,et al.  Integrating artificial neural networks with rule-based expert systems , 1994, Decis. Support Syst..

[68]  I. Benbasat,et al.  Individual-Differences In The Use Of Decision Support Aids , 1982 .

[69]  Daniel J. Power,et al.  AN EMPIRICAL ASSESSMENT OF COMPUTER‐ASSISTED DECISION ANALYSIS* , 1986 .

[70]  Julie E. Kendall,et al.  The relationship of organizational subcultures to DSS user satisfaction , 1987 .

[71]  E. Balagurusamy,et al.  Expert Systems for Management and Engineering , 1991 .

[72]  M. Lynne Markus,et al.  Power, politics, and MIS implementation , 1987, CACM.

[73]  Robert P. Bostrom,et al.  Personality Characteristics of MIS Project Teams: An Empirical Study and Action-Research Design , 1982, MIS Q..

[74]  Kathy Brittain White,et al.  Information Systems Development Success: Perspectives from Project Team Participants , 1986, MIS Q..

[75]  Alexander M. Maish A User's Behavior Toward His MIS , 1979, MIS Q..

[76]  Edward A. Feigenbaum,et al.  The rise of the expert company , 1988 .

[77]  Craig K. Tyran,et al.  The implementation of expert systems: a survey of successful implementations , 1993, DATB.

[78]  Dennis F. Galletta,et al.  Some Cautions on the Measurement of User Information Satisfaction , 1989 .

[79]  Donald A. Waterman,et al.  A Guide to Expert Systems , 1986 .