Benefit-of-the-doubt approaches for calculating a composite measure of quality

Standard approaches for determining weights when calculating a composite measure of health care quality from individual quality indicators (QIs) include equal weighting, opportunity-based weights, and judgment-based weights. Benefit-of-the-doubt approaches have not been used in the health services area, though one has been used to calculate composite measures for profiling countries. Underlying these approaches is the assumption that relative performance on a set of indicators is, at least to some extent, a revealed preference by the organizational unit about the relative importance of the indicators. A benefit-of-the-doubt approach recognizes these revealed preferences by assigning higher weights to indicators on which performance is better and lower weights to indicators on which performance is poorer. We consider two benefit-of-the-doubt approaches. The first uses simple linear programming (LP) models; the second uses data envelopment analysis (DEA), the way in which the benefit-of-the-doubt approach has been previously implemented. In both cases, constraints are added to limit weight adjustments to some percentage of policy-determined baseline weights. Using both standard and benefit-of-the-doubt approaches, composite scores are calculated from data on five QIs from 32 Department of Veterans Affairs (VA) nursing homes. We examine the tradeoff between the level of allowable weight adjustment and impact on facility rankings. If weights are constrained to be within 75% of baseline weights, all approaches identify pretty much the same high performing facilities. Weights from benefit-of-the-doubt approaches, because they are able to reflect local preferences and conditions, should be attractive to facilities and, in a collaborative environment, to policy makers.