The Role of Automation in Complex System Failures

Although proponents of advanced information technology argue that automation can improve the reliability of health care delivery, the results of introducing new technology into complex systems are mixed. The new forms of failure that accompany automation challenge technical workers, often demanding novel approaches to recovering from failure and restoring system operations. We present a case where automation created a new form of failure that was not foreseen. In this case, human practitioners were the main source of robust and reliable health care delivery and were the primary agents in recovering from a life-threatening automation failure. The features of this case correspond closely to experience with new information technology in other domains and have implications for plans to improve patient safety using technology.

[1]  K. Weick.  The Vulnerable System: An Analysis of the Tenerife Air Disaster , 1990 .

[2]  Richard I. Cook.  Inside risks: Observations on RISKS and risks , 1997, CACM.

[3]  K. Mellanby.  How Nature works , 1978, Nature.

[4]  R. M. Alexander,et al.  Sources of Power , 1982 .

[5]  Richard I. Cook,et al.  SPECIAL SECTION: Adapting to New Technology in the Operating Room , 1996, Hum. Factors.

[6]  Anita L. Tucker,et al.  Why Hospitals Don't Learn from Failures: Organizational and Psychological Dynamics That Inhibit System Change , 2003 .

[7]  Richard I. Cook.  Observations on RISKS and Risks. , 1997 .

[8]  Emilie M. Roth,et al.  Cognitive Engineering: Human Problem Solving with Tools , 1988 .

[9]  K N Barker,et al.  Effect of an automated bedside dispensing machine on medication errors. , 1984, American journal of hospital pharmacy.

[10]  D M Gaba,et al.  Case 2-1992. Unintentional delivery of vasoactive drugs with an electromechanical infusion device. , 1992, Journal of cardiothoracic and vascular anesthesia.

[11]  G. Klein,et al.  Decision Making in Action: Models and Methods , 1993 .

[12]  D. Richard Kuhn,et al.  Sources of Failure in the Public Switched Telephone Network , 1997, Computer.

[13]  Christopher Nemeth,et al.  Crafting information technology solutions, not experiments, for the emergency department. , 2004, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[14]  Earl L. Wiener,et al.  Human factors of advanced technology (glass cockpit) transport aircraft , 1989 .

[15]  D. Woods.  Coping with complexity: the psychology of human behaviour in complex systems , 1988 .

[16]  Christopher Nemeth,et al.  The Messy Details: Insights From the Study of Technical Work in Healthcare , 2004 .

[17]  Richard I. Cook,et al.  Lost in menuspace: user interactions with complex medical devices , 2004, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[18]  Anita L. Tucker,et al.  When Problem Solving Prevents Organizational Learning , 2002 .

[19]  Roberta Calderwood,et al.  Critical decision method for eliciting knowledge , 1989, IEEE Trans. Syst. Man Cybern..

[20]  Gavriel Salvendy,et al.  Handbook of Human Factors and Ergonomics , 2005 .

[21]  Thomas B. Sheridan,et al.  Healthcare safety: the impact of Disabling "safety" protocols , 2004, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[22]  Colin Potts,et al.  Design of Everyday Things , 1988 .

[23]  James T. Reason,et al.  Managing the risks of organizational accidents , 1997 .

[24]  J. E. Groves,et al.  Made in America: Science, Technology and American Modernist Poets , 1989 .

[25]  D. Woods,et al.  Automation Surprises , 2001 .

[26]  D M Berwick.  You cannot expect people to be heroes. , 1998, BMJ.

[27]  S D Berns,et al.  The potential for improved teamwork to reduce medical errors in the emergency department. The MedTeams Research Consortium. , 1999, Annals of emergency medicine.

[28]  Charles E. Billings,et al.  Aviation Automation: The Search for A Human-centered Approach , 1996 .