Does the inter-unit reliability (IUR) measure reliability?

In monitoring health care providers, various outcomes are used to assess the performance and quality of care given. We consider a measure that is normally distributed across the majority of providers with both a within and between component contributing to the overall variability of the measure. In such cases, the inter-unit reliability (IUR) is commonly used to assess the usefulness of a measure for identifying extreme providers. In this article, we define and discuss the IUR and note its role under various assumptions about the source of the between-provider variance. This variability may be due primarily to differences in the patients that are not accounted for in the measured covariates, or differences in the quality of care provided, or a combination of these two. The IUR is a simple population characteristic specifying the proportion of the variation in the measure that is related to between-provider differences without regard to the source of that variation. On the other hand, the IUR does not characterize the suitability of a measure for profiling or identifying outliers except in very special circumstances where all the variation is due to quality of care and there are no outliers. In assessing the reliability of a measure for profiling, the key question is whether outlying providers with poor (or excellent) quality of care give rise to extreme values of the measure.