Acceptability and feasibility of the Leapfrog computerized physician order entry evaluation tool for hospitals outside the United States

BACKGROUND Computerized physician order entry (CPOE) with clinical decision support is expected to deliver many benefits in terms of patient safety. The Leapfrog tool was developed to allow hospitals to assess their medication-safety related decision support. To explore the approach's generalizability, we examined its acceptability and feasibility through an evaluation using this tool in four Korean hospital systems. METHODS Four hospitals with locally developed CPOE systems participated, and a cross-sectional study design was used with the approval of the Leapfrog Group and the institutional review board at each hospital site. The hospitals were tertiary and academic institutions with long experience of advanced CPOE. From January 21 to 28, 2014, web-based tests were conducted at each site following the given instructions, and the results were self-reported. We measured each system's response rate, category completion rate, and time to complete the evaluation. Additionally, we compared the evaluation results of the four systems with scores from five US systems, as was reported in another study. RESULTS The four systems underwent the tests, and the overall category completion rates ranged from 67.9% to 75.5%. The times to finish the tests were tolerable and within the allowed test timeframe. One system did not pass the deception analysis, which checks for false positives, due to a conflict with another type of alert checking for the presence of a medical diagnosis for documentation purposes. The other three systems scored at the completed the evaluation stage, with scores ranging from 21.6% to 36.5%. Of the nine error categories, Drug-Allergy was an area of strength for all systems, whereas the categories of Therapeutic duplication, Drug-Labs, and Drug-Age were areas of weakness for all. In comparison with the US systems, gaps were identified, and further improvement is needed. CONCLUSIONS The acceptability of the CPOE evaluation tool was moderate, but the feasibility was sufficient to operate the test outside the US, and the results revealed many opportunities for improvement in the Korean systems, as was the case when this application was introduced in the US.