Expert systems (ES) were among the earliest branches of artificial intelligence (AI) to be commercialized. But how successful have they actually been? Many well-publicized applications have proven to be pure hype, numerous AI vendors have failed or been completely reorganized, major companies have reduced or eliminated their commitment to expert systems, and even Wall Street has become disillusioned--a predicted $4 billion market proving to be smaller by an order of magnitude. Yet, in spite of these setbacks, there are many companies who remain enthusiastic proponents of the technology and continue to develop important ES applications. This paper explores how the first wave of commercial expert systems, built during the early and mid-1980s, fared over time. An important subset of these systems, identified in a catalog of commercial applications compiled in 1987, was located through a telephone survey, and detailed information on each system was gathered. The data collected show that most of these systems fell into disuse or were abandoned uring a fiveyear period from 1987 to 1992, while about a third continued to thrive. Quantitative and qualitative analysis of the data further suggests that the short-rived nature of many systems was not attributable to failure to meet technical performance or economic objectives. Instead, managerial issues such as lack of system acceptance by users, inability to retain developers, problems in transitioning from development to maintenance, and shifts in organizational priorities appeared to be the most significant factors resulting in long-term expert system disuse. Introduction Expert systems, one of the earliest branches of artificial intelligence (AI) to achieve widespread commercial viability, present managers with a paradox. The technology, which applies AIderived specialized symbolic reasoning techniques to solve difficult problems (Luconi, et al., 1986), produced a series of resounding successes in the early and mid-1980s. Well known systems, such as Digital’s XCON, Coopers and Lybrand’s ExpertTax, and American Express’ Authorizer’s Assistant, have amply demonstrated the technology’s capability both to generate huge financial returns and to contribute to the strategic goals of the firm (Sviokla, 1986). By the late 1980s, however, another attitude toward expert systems began to surface in the AI and management communities. Critics argued that expert systems, as a class, rarely succeed or, perhaps, cannot succeed in delivering expert performance. ~ Wall Street, once enthusiastic about the prospects of expert systems, became suspicious of a technology that repeatedly failed to deliver on its promises. As the Wall Street Journal reported: The AI industry, which many market researchers had predicted would reach $4 billion annual sales by now, remains nascent. Generous estimates of the market today are closer to $600 million. After swallowing up hundreds of millions of dollars in venture capital and exciting some of the brightest professors at top technical schools with visions of riches, hundreds of AI start-ups have yielded only a few profitable public companies (Bulkeley, 1990, Section B, p. 1). Even industry participants have voiced serious concerns. In a recent survey, 60 percent predicted that the AI industry would either remain flat or decline between 1993 and 1999 (Coleman, 1993). At the present time, considerable divergence of opinion exists regarding how well expert systems have fared. The travails of AI vendors, who are important suppliers of expert system tools, suggest that demand for ES technology is not exactMIS Quarterly/March 1995 51 Early Expert Systems: Update ly thriving. Som~ of the most influential hardware and tool companies (e.g., Gold.Hill, Intellicorp, Inference, Teknowledge, and Symbolics) have been forced to reorganize, enduring major cutbacks in the process. Other participants have simply gone out of business (e.g., Palladian and Lisp Machines, Inc.). Broadly based companies that formerly maintained AI divisions or products (e.g., Texas Instruments, Xerox, Borland, Microsoft, and Radian) have refocused their efforts elsewhere. Not all indicators of ES technology are bleak, however. Conversations with senior managers indicate that a number of major cbmpanies, such as Digital Equipment Corp., Coopers & Lybrand, and American Express, persist in actively developing and maintaining expert systems. Some, in fact, assert that their ,key businesses are strategically dependent on these systems and are likely to remain so in the future. Furthermore, a variety of new products, ranging from tax preparation software t.o music and language instruction systems to college search software, now incorporate ES technology. While numerous opinions exist regarding how successful commercial expert systems have been, the basis for these opinions is largely anecdotal. What is almost completely lacking is quantitative data addressing the question of how well commercial expert systems, as a group, have fared. Specifically, we know little about how the first wave of commercial ES applications performed, or how many are still in use today. Further, almost no information has been systematically gathered to identify those factors that influenced usage. This paper describes a field study that examined how these early systems fared. In particular, the study acquired and analyzed.data relating to two important measures of ES use: ̄ User penetration--the degree to which potential users became actual users, and ̄ Longevitymthe period of time over which the system was used. Its specific goals were .to better understand how system performance, system economics, and organizational factors contributed to early ES use. The approach entailed locating approximately 80 expert systems that were developed in the early and mid-1980s and then gathering data relating to the following questions: 1. To what extent did issues of system performance appear to affect the levels of user penetration and longevity of the systems examined? 2. To what extent did the economics of development and maintenance appear to affect the levels of user penetration and longevity of the systems examined? 3. To what extent did issues of individual and organizational adoption, such as user acceptance and fit with organizational priorities, appear to affect the levels of user penetration and longevity of the systems examined? These questions were deemed particularly relevan.t to managers who oversee the development of expert systems and must therefore decide how to prioritize their company’s efforts and resources. Expert Systems Development: Alternative Perspectives The question of how to build and implement expert systems has been studied extensively. Two distinct perspectives have emerged from the literature: 1. A technical perspective, which emphasizes the technological, managerial, and economic issues associated with constructing ES applications that deliver appropriate performance in a timely and cost-effective manner. 2. An organizational perspective, which particularly concerns itself with the challenges of managing the process of deploying and using systems within an organizational setting. The two perspectives are reviewed in the next two sections. ES development: technical perspective Unlike conventional systems, which have existed since the mid-1950s, commercial expert systems have been around for a very short time--few were constructed more than a decade ago. As a consequence, much of the research on building these systems has focused on technical and software development issues. This emphasis is reflected in the types of research that have re52 MIS Quarterly/March 1995 Early Expert Systems: Update ceived the greatest attention in the ES literature, including: ̄ Identifying task and domain areas that are suitable for ES (e.g., Buchanan, et al., 1983; Harmon, et al., 1988; Harmon and King, 1985; Prereau, 1985; Silverman, 1987; Stefik, et al., 1983; Walker and Miller, 1990; Waterman, 1986). ̄ Deciding whether or not to build an ES (e.g., Harmon and Sawyer~ 1990; Silverman, 1987; Turban, 1992; Waterman, 1986). ̄ Selecting appropriate ES tools (e.g., Gevarter, 1987; Gill, 1991; Harmon, et al., 1988; Harmon and Sawyer, 1990; Stefik, et al., 1983; Stylianou, et al., 1992; Waterman, 1986; Waterman and Hayes-Roth, 1983). ̄ Roles and stages in ES development (e.g., Buchanan, et al., 1983; 1986; Harmon and Sawyer, 1990; Turban, 1992; Waterman, 1986). ̄ Working with domain experts and knowledge acquisition (e.g., Bobrow, et al., 1986; Harmon and King, 1985; Harmon and Sawyer, 1990; Prereau, 1985; Slatter, 1987; Waterman, 1986). ̄ Integrating AI/ES and conventional technologies (e.g., Freedman, 1987; Freundlich, 1990; Harmon and Sawyer, 1990; Stapleton, 1988; Turban and Watkins, 1986). Beyond the ES development literature, there is also an immense computer science-grounded ES literature, addressing topics such as knowledge and uncertainty representation, algorithms and inference engine design, and automated knowledge acquisition. Faithful to its focus on design and development, the technical perspective emphasizes, as its number one objective, building systems that exhibit high performance. This is loosely defined to mean systems that "successfully solve the problems to which they are applied" (Brachman, et al., 1983, p. 44). Many different criteria reflecting overall system performance xist, however. Among these are: 1. Consistency: The system’s ability to produce task solutions with a level of consistency as great or greater than that of the expert. Where expertise exists in multiple individuals, such performance may be reflected in greater consistency than previously existed between experts. For example, one of the performance characteristics cited for Texas Instruments’ Capital Expert (Gill, 1987) was its ability to enforce consistency in the preparation of capital expenditure proposals across the company. 2. Quality
[1]
Perspective
,
1974
.
[2]
Frederick Hayes-Roth,et al.
An investigation of tools for building expert systems
,
1982
.
[3]
Roger S. Pressman,et al.
Software Engineering: A Practitioner's Approach
,
1982
.
[4]
Frederick Hayes-Roth,et al.
Building expert systems
,
1983,
Advanced book program.
[5]
Lawrence Birnbaum,et al.
The architecture of expert systems
,
1983
.
[6]
Paul Harmon,et al.
Expert systems: artificial intelligence in business
,
1985
.
[7]
David S. Prerau.
Selection of an Appropriate Domain for an Expert System
,
1985,
AI Mag..
[8]
Alex Goodall,et al.
The guide to expert systems
,
1985
.
[9]
Clive L. Dym.
Building expert systems: F. Hayes-Roth, D. A. Waterman and D. B. Lenat, (Addison-Wesley, Reading, MA, 1983); 444 pages, $32.50
,
1985
.
[10]
Daniel G. Bobrow,et al.
Expert systems: perils and promise
,
1986,
CACM.
[11]
Patrick Henry Winston,et al.
Amplifying Expertise with Expert Systems
,
1986
.
[12]
Thomas W. Malone,et al.
Expert systems: the next challenge for managers
,
1986
.
[13]
Bruce C. Buchanan,et al.
Expert Systems
,
1988,
J. Autom. Reason..
[14]
Efraim Turban,et al.
Integrating Expert Systems and Decision Support Systems
,
1986,
MIS Q..
[15]
William B. Gevarter.
The Nature and Evaluation of Commercial Expert System Building Tools
,
1987,
Computer.
[16]
Barry G. Silverman.
Should a manager “hire” an expert system
,
1987
.
[17]
Philip E. Slatter,et al.
Building expert systems: cognitive emulation
,
1987
.
[18]
Barry G. Silverman.
Expert systems for business
,
1987
.
[19]
Paul Harmon,et al.
Expert systems: tools and applications
,
1988
.
[20]
Vice President,et al.
An Introduction to Expert Systems
,
1989
.
[21]
Yehudah Freundlich.
Transfer Pricing-integrating expert systems in MIS environments
,
1990,
IEEE Expert.
[22]
Paul Harmon,et al.
Creating Expert Systems for Business and Industry
,
1990
.
[23]
Knowledge-based systems: Markets, suppliers and products : Ovum Ltd
,
1990,
Knowl. Based Syst..
[24]
Chyck Karr,et al.
Applying genetics to fuzzy logic
,
1991
.
[25]
Thomas Grandon Gill.
Expert systems: a mapping between symbol susceptible tasks and software tools
,
1991
.
[26]
Efraim Turban,et al.
Expert systems and applied artificial intelligence
,
1992
.
[27]
Ephraim R. McLean,et al.
Information Systems Success: The Quest for the Dependent Variable
,
1992,
Inf. Syst. Res..
[28]
Robert D. Smith,et al.
Selection criteria for expert system shells: a socio-technical framework
,
1992,
CACM.
[29]
Gautam Biswas,et al.
Extending component connection modeling for analyzing complex physical systems
,
1993,
IEEE Expert.
[30]
Craig K. Tyran,et al.
The implementation of expert systems: a survey of successful implementations
,
1993,
DATB.
[31]
John Durkin,et al.
Expert Systems
,
1994
.