Generic Health Measurement: Past Accomplishments and a Measurement Paradigm for the 21st Century

Over the past 30 years, researchers have generated numerous tools that use self-reporting to measure functional status, emotional well-being, and subjective perceptions of health [1-22]. Uses of generic surveys have increased dramatically in recent years as a result of the outcomes movement. Some authors have advocated the routine inclusion of data on generic health status in large databases [23], but others, such as Liang and Shadick [24], caution that the utility of doing so remains largely unproven. The purpose of this paper is to describe the history of generic health measurement and to suggest a more modern measurement paradigm for the 21st century. The conjoint use of computerized adaptive testing and item response theory offers distinct advantages for health outcomes assessment that could improve the feasibility and utility of including patient-centered data in large administrative databases. The Evolution of Generic Health Measurement Figure 1 presents a timeline of the evolution of generic health measures with respect to broader developments in health policy and health status assessment. Roughly coincidental with the publication of the World Health Organization's definition of health [25] was the emergence of clinically based, global rating scales whose content extended beyond organ function to encompass human function. Such measures as the Karnofsky performance status scale [26] and the function scale of the American Rheumatoid Association [27] were intended to supplement physiologic measures in an attempt to better understand treatment effectiveness. Around the same time, efforts to modernize national health indicators, including the incorporation of single-item indicators of activity limitations and perceived health in the National Health Interview Survey [28], were initiated [29]. Figure 1. Timeline of the evolution of generic health measures with respect to broader developments in health policy and health status assessment. The policy initiatives of the War on Poverty in the mid-1960s prompted two advances in health measurement. First, the social indicators movement ushered in measurement of quality of life in general populations [30, 31] and provided indicators of how well we lived, which were to be used with existing measures of how much we produced and spent [32]. Second, unified indexes of mortality and morbidity were developed for planning and evaluation purposes at the population health level [33-35]. A watershed for generic health assessment can be traced to the Human Population Laboratory, which launched measurement work in physical, mental, and social health [1-4]. As important, the Human Population Laboratory demonstrated that respondents will complete long surveys by mail [36], a finding that reduced the bias against mail surveys. In the 1970s, the development of generic tools proliferated, in part as a result of extramural support from the National Center for Health Services Research. Definitional expansiveness was the signature of this era, and multi-item scales replaced single-item measures. The Quality of Well-Being Scale, developed for priority setting and program evaluation, represented a meaningful advance by measuring the value components of a social indicator of health [6, 37]. Next, the Sickness Impact Profile [7, 38] was developed for health care evaluation. The 136 items in this profile were obtained from patients, providers, and caregivers and yielded individual health profiles and summary scores. The McMaster Health Index Questionnaire [8, 39] followed. Intended for use in clinical and health services research, it measured physical, social, and mental health by using 59 items. The Health Perceptions Questionnaire [40] was constructed for use in health planning and evaluation and tapped the elusive realm of positive health. In 1979, health status measures for the adult general population emerged from the Health Insurance Experiment [9]. Next, the Nottingham Health Profile [10, 41] was developed for use in population surveys, clinical trials, and clinical practice. The 38 items in the Nottingham Health Profile tapped six health concepts and were derived from patients. The Duke Health Profile [11] was developed for use in research and clinical applications in primary care. The 63 items in this profile covered four health concepts and were obtained from the literature. In the early 1980s, development of new measures took a respite but health research increasingly applied existing measures [42-44]. Interest in methodologic issues increased [45-48]. By the mid-1980s, interest had developed in the use of generic tools in everyday clinical practice, largely because of research showing poor correspondence between clinician and patient ratings of function and well-being [49-51]. In addition, growing recognition of the biopsychosocial model [52] and its relevance to an aging population resulted in increased appreciation that the preservation of function and well-being is an important goal of medical care [53]. Clinical practice applications ushered in the era of practicality. Shorter tools were developed: The Functional Status Questionnaire consisted of 34 items [13], and the Dartmouth COOP Charts had 9 items [14]. These tools were developed with measurement priorities directed toward practical efficiency (for example, ease of administration and scoring), which was achieved at the expense of measurement precision [54, 55]. The most recent era of health measurement is that of psychometric efficiency, which has several underpinnings. First, the outcomes movement gained momentum after Ellwood's Shattuck lecture was published [23] and the Agency for Health Care Policy and Research was established in 1989. Large-scale studies of patient-based outcomes were imminent. Second, burdened by study costs that spanned outcomes ranging from pathophysiology to quality of life, the clinical trials community sought more economical measures of health status. Third, concerns about respondent burden among severely ill patients encouraged shorter surveys. The Medical Outcomes Study (MOS) Short Form (SF) 20 Survey [16, 56] was the first to surface. The 20 items derived largely from the Health Insurance Experiment and tapped six health concepts. Next emerged the Duke Health Profile [17], a 17-item survey that was empirically derived from the original Duke Health Profile. The SF-36 [21] developed out of the SF-20 and the 149-Item Functioning and Well-Being Profile, which measures 16 health concepts [19]. The SF-6 Survey, derived from the Functioning and Well-Being Profile, uses a single item to tap 6 health concepts [19]. The SF-12 Survey is an empirically derived short form of the SF-36 [22]. Over the past 30 years, we have greatly improved our measurement bandwidth in generic health assessment (the breadth of health dimensions measured). Many different health concepts are now measured across the armamentaria of generic tools, although specific surveys differ in bandwidth (for example, the Sickness Impact Profile measures 12 health concepts, whereas the McMaster Health Index Questionnaire measures just 3). However, many generic measures, even those with excellent bandwidth, still have problems of fidelity (that is, thoroughness and depth of measurement). Thus, although we now quantify many different dimensions of health, we often do so at the expense of precision. Overall, many generic tools lack the precision required for effective health care decision making. Precision is conceptualized here and elsewhere [57] as a property of a measure that encompasses both the range or depth of measurement and the number of distinct levels enumerated by a scale (fineness of specification). Prevailing Measurement Paradigm Generic health status tools have been developed in the group-testing tradition. The defining signature of group tests is the use of a fixed set of questions (items) for all respondents, regardless of the appropriateness of any specific item for a given individual respondent. Items in group tests are selected or written to represent a moderate range of activities at a moderate level of difficulty. The era of psychometric efficiency emphasized construction of generic measures containing as few items as possible. Acceptable standards of face and content validity and reliability with few items can best be achieved by selecting items that are fairly homogeneous. Thus, selected items are often in the middle range of item difficulty and are almost alternate forms of each other. These measurement standards have two consequences, both of which are evident in generic measures. First, fixed-length health surveys tend to bore healthier respondents (because they have to wade through items that are easy for them to do, such as bathing) and frustrate more impaired respondents (because they have to respond to items that are clearly impossible for them to do, such as running one block). Such complaints about generic surveys are common from respondents. Respondents do not object to survey length itself; rather, they are frustrated by redundant items and items that to them are of low salience and relevance [58-60]. Second, because item selection is geared toward the middle-of-the-road in content coverage and difficulty, the end points of the health continuum tend to be poorly defined. This yields ceiling effects for general populations and floor effects for more disabled populations. For many generic measures [54, 55, 61], score distributions are often highly skewed, such that a plurality of respondents are classified as being in a state of perfect health at or near to the ceiling of the scale. Very large ceiling effects (up to 70%) have been observed in general and primary care populations [41, 62-64]. Ceiling effects are more prevalent than floor effects because many generic tools represent health as the absence of limitations. Score imprecision has two principal consequences. First, it is impossible to distinguish a