Summary Recommendations for Responsible Monitoring and Regulation of Clinical Software Systems

Health care practitioners, clinical facilities, industry, and regulatory agencies share an obligation to manage clinical software systems responsibly by using a common framework. Clinical software systems are defined as algorithmic programs, related knowledge bases, and embedded interfaces that directly contribute to the delivery of health care as opposed to inventory or accounting functions. Clinical software systems are ubiquitous, and a growing literature documents how these systems can improve health care processes and outcomes. Although no reports suggest that use of clinical software systems causes substantial harm to patients, concerns about patient safety must be addressed. In mid-1996, the U.S. Food and Drug Administration (FDA) called for discussions on the regulation of software programs as medical devices [1]. A 1989 draft policy [2], never adopted formally, served to guide FDA conduct. That draft policy recommended that the FDA exempt from regulation generic software (that is, content-free spreadsheet and database programs), educational systems that merely provide information, and systems that generate advice for clinician-users in a manner that users can easily override. In response to the 1996 announcement by the FDA, a consortium of health information-related organizations developed recommendations for public and private actions that are intended to responsibly monitor and regulate clinical software systems. For a list of consortium organizations, see the Appendix. We present a concise statement of the consortium's position. A more thorough and technically oriented version of this manuscript is currently in press elsewhere [3]. That document describes benefits of clinical software systems, obstacles to evaluating and monitoring such systems, justifications for each consortium recommendation, information on participating organizations, and a summary of the consensus process. That report also includes an extensive bibliography. The Complexity of Clinical Software Systems Because clinical software systems are diverse and complex, it is difficult to determine their safety. Thousands of clinical software products compete in the commercial marketplace. Many home-grown systems exist, and biomedically oriented World Wide Web sites of variable quality proliferate in an uncontrolled manner. Many installations are unique. Significant functional changes occur when a software product is integrated locally into a clinical information management infrastructure. Upgrades and maintenance increase the functionality, variety, and complexity of clinical systems. Finally, interactions between clinical software programs and individual users may cause unpredictable outcomes that are not related to software malfunctions. Past and Current Regulation of Clinical Software Systems Through its mandate from the U.S. Congress to safeguard the public, the FDA has regulated marketing and use of medical devices. Section 201(h) of the 1976 Federal Food, Drug, and Cosmetic Act [4] defines a medical device as any instrument, apparatus, implement, machine, contrivance, implant, in vitro reagent, or other similar or related article, including any component, part, or accessory, which is intended for use in the diagnosis of disease or other conditions, or in the cure, mitigation, treatment, or prevention of disease or intended to affect the structure or any function of the body. Representatives for the FDA have stated that clinical software programs, whether they are associated with biomedical devices or stand alone, are contrivances and, therefore, fall within the FDA's realm of responsibility. The FDA regulates medical devices that are 1) commercial products used in patient care, 2) devices used in the preparation or distribution of clinical biological materials [such as blood products], or 3) experimental devices used in research involving human participants. Commercial vendors of specified types of medical devices must register as manufacturers and list their devices with the FDA. Upon listing, the FDA classifies medical devices by category. In its regulation of classified medical devices, the FDA usually takes one of three courses of action. First, the FDA can exempt specific devices or categories of devices that are deemed to pose minimal risk. Second, the FDA uses the 510(k) process (premarket notification) for nonexempt systems. Through the 510(k) process, manufacturers attempt to show that their devices are equivalent in purpose and function to low-risk (FDA class I or class II) devices that have already been approved by the FDA or to devices marketed before 1976. Such devices can be directly cleared by the FDA. Finally, the FDA requires premarket approval for higher-risk (FDA class III) products and products with new (unclassified) designs invented after 1976. Through the premarket approval process, a manufacturer provides evidence to the FDA that a product performs its stated functions safely and effectively. Premarket approval is especially important for products that pose substantial potential clinical risk. Premarket notification and premarket approval usually take a few to many months to complete and may involve numerous iterations. Exemption can take place in two ways: A device can be exempt from registration and thus not subject to 510(k) requirements at all, or a category of listed (classified) devices may be specifically exempted from certain regulatory requirements. When a nonexempt product is modified substantially (as defined by FDA guidelines), the vendor must reapply to the FDA for clearance through the 510(k) or premarket approval mechanisms. Consortium Recommendations The consortium believes that users, vendors, and regulatory agencies, including the FDA, should adopt the recommendations listed below. Several of these recommendations involve innovations that should be developed and tested on a limited scale before they are widely adopted. Detailed analysis of the potential effects on health care providers, patients, and vendors should precede implementation of any new procedures for the regulation and monitoring of clinical software systems. At the same time, it is important that manufacturers, users, and patients not be required to tolerate delays in critical improvements to software that might result from excessive governmental or local review and approval processes. Recommendation 1 The consortium proposes four categories of clinical software system risks and four classes of measured regulatory actions as a template for determining how to monitor or regulate any clinical software system (Table 1). Decisions about whether to install and how to monitor clinical software systems should take into account 1) the clinical risks posed by software malfunction or misuse; 2) the extent of system autonomy from user supervision and control [that is, the lack of opportunity for qualified users, such as licensed practitioners, to recognize and easily override clinically inappropriate recommendations or other forms of substandard software performance]; 3) the pattern of distribution and degree of support for the software system, including local customization; 4) the complexity and variety of clinical software environments at installation sites; 5) the likelihood that systems and their environments will evolve and change over time; and 6) the ability of proposed monitors or regulators to detect and correct problems in a manner that protects patients. Recommendation 2 Clinical software systems should be supervised locally if possible through the creation of software oversight committees (Table 1). Table 1. Consortium Recommendations for Monitoring and Regulating Clinical Software Systems* Local software installation sites have the greatest capability to detect software problems, analyze their effects, and develop timely solutions. It would be advantageous to develop a program of institutional- and vendor-level controls for most clinical software products rather than to mandate universal, national-level monitoring, which is likely to be cumbersome, inefficient, and costly. Institutional review boards are a federally mandated local monitoring process for the conduct of clinical research. Like institutional review boards, software oversight committees would serve as guardians to ensure that institutional research processes that affect patients are conducted properly. However, the analogy between the software oversight committee and the institutional review board is not complete because the responsibilities of the two groups differ. Software oversight committees would monitor processes that are far less discrete than formal research protocols. To protect patient safety and ensure that software programs do not disrupt the integrity of clinical practice, local software oversight committees should enlist members with expertise in health care informatics, health care delivery, data quality, biomedical ethics, patient perspectives, and quality improvement. Software oversight committees should work with system administrators, users, and vendors to ensure that appropriate ongoing monitoring is in place to detect and address adverse events and that the overall system performs as designed. They might apply pressure to correct vendor-product related problems when the usual means of customer-vendor dialogue fail. Although software oversight committees need not install software or monitor software functions directly, they must ensure that others do so in a manner that protects patient safety and institutional integrity. It will be important to specify ethical guidelines for conduct of the software oversight committee and rules for avoiding conflicts of interest between local employees (including members of software oversight committees) and commercial vendors (including employee-owned enterprises). Recommendation 3 Budgetary, legal, and logistic constraints limit the type and number of systems that the FDA can regulate effectively. The F