Computerized physician order entry has been shown to reduce the frequency of serious medication errors. Decision support tools such as alerting functions for patient medication allergy are a key part of these applications. However, optimal performance requires iterative refinement. As systems become increasingly complex, mechanisms to monitor their performance become increasingly critical. We analyzed trend data obtained over a five-year period that showed decreasing compliance to allergy alert functions within computerized order entry. Many medication-allergy pairs were being consistently overridden. Renewal policies affecting reordering narcotics also contributed heavily to this trend. Each factor revealed a system-wide trend that could result in suggestions for policy or software change. Monitoring trends such as these is very important to maintain software correctness and ensure user trust in alerting systems, so users remain responsive to computerized alerts.