Computerized Physician Order Entry (CPOE) Systems can reduce the number of medication errors and Adverse Drug Events (ADEs). However, studies have shown that users often override alerts, as they feel these are too unspecific for the given patient context. It is unclear, however, how alerts could be contextualized, that is adapted to the clinical context. Based on a literature search, we developed a list of 20 possible context factors. We asked 69 international CPOE researchers and 120 physicians from four hospitals in two countries to judge the usefulness of each factor. Researchers judged the following factors as most important: 1.) Severity of the effect, 2.) Clinical status of the patient, 3.) Probability of occurrence, 4.) Risk factors of the patient, 5.) Strength of evidence. Physicians judged the following factors as most important: Severity of the effect, clinical status of the patients, complexity of the case, and class of drug. These top-ranked context factors could be used to re-design the way alerts are presented in CPOE systems, to increase sensitivity of alerts, to reduce overriding rates, and to improve medication safety.