Is the problem list in the eye of the beholder? An exploration of consistency across physicians

OBJECTIVE Quantify the variability of patients' problem lists - in terms of the number, type, and ordering of problems - across multiple physicians and assess physicians' criteria for organizing and ranking diagnoses. MATERIALS AND METHODS In an experimental setting, 32 primary care physicians generated and ordered problem lists for three identical complex internal medicine cases expressed as detailed 2- to 4-page abstracts and subsequently expressed their criteria for ordering items in the list. We studied variability in problem list length. We modified a previously validated rank-based similarity measure, with range of zero to one, to quantify agreement between pairs of lists and calculate a single consensus problem list that maximizes agreement with each physician. Physicians' reasoning for the ordering of the problem lists was recorded. RESULTS Subjects' problem lists were highly variable. The median problem list length was 8 (range: 3-14) for Case A, 10 (range: 4-20) for Case B, and 7 (range: 3-13) for Case C. The median indices of agreement - taking into account the length, content, and order of lists - over all possible physician pairings was 0.479, 0.371, 0.509, for Cases A, B, and C, respectively. The median agreements between the physicians' lists and the consensus list for each case were 0.683, 0.581, and 0.697 (for Cases A, B, and C, respectively).Out of a possible 1488 pairings, 2 lists were identical. Physicians most frequently ranked problem list items based on their acuity and immediate threat to health. CONCLUSIONS The problem list is a physician's mental model of a patient's health status. These mental models were found to vary significantly between physicians, raising questions about whether problem lists created by individual physicians can serve their intended purpose to improve care coordination.