Development and Validation of a Multifactorial Risk Index for Predicting Postoperative Pneumonia after Major Noncardiac Surgery

Postoperative pulmonary complications are associated with substantial morbidity and mortality. It has been estimated that nearly one fourth of deaths occurring within 6 days of surgery are related to postoperative pulmonary complications (1). Postoperative infections are also a major source of the morbidity and mortality associated with undergoing surgery. Pneumonia is the most serious postoperative complication that is included in both of these categories. Pneumonia ranks as the third most common postoperative infection, behind urinary tract and wound infection (2). According to the National Nosocomial Infection Surveillance system, pneumonia occurred in 18% of patients after surgery (3). Postoperative pneumonia occurs in 9% to 40% of patients, and the associated mortality rate is 30% to 46%, depending on the type of surgery (1, 4). Previous studies of risk factors used various definitions of postoperative pulmonary complications. Atelectasis (1, 47), postoperative pneumonia (12, 46, 811), the acute respiratory distress syndrome (9, 12), and postoperative respiratory failure (6, 9, 11, 13) have been classified as postoperative pulmonary complications. Although the clinical significance of each of these complications varies greatly, they were grouped together as a single outcome in previous studies (6). Some studies were limited to examination of risk factors in patients undergoing abdominal or thoracic procedures or in patients with specific medical conditions, such as chronic obstructive pulmonary disease (2, 4, 6, 1012, 14). These studies were often based on a small sample from one institution, and studies of independent samples did not validate their findings (15, 16). Although clinicians have used preoperative cardiac risk indexes for more than 20 years (17), previous studies have not produced a validated risk index that predicts risk for pneumonia (18). Our main objective was to develop and validate a multifactorial risk index for predicting postoperative pneumonia. Preoperative risk factors for postoperative pneumonia were determined in a large multicenter observational cohort of patients undergoing various major noncardiac operations. Specific goals included identification of predictors that are easily obtained and commonly accessible to care providers before surgery. Using these predictors, we developed and validated a risk assessment model and scoring system for predicting postoperative pneumonia, analogous to risk assessment models for predicting cardiac complications (17, 19, 20). Methods Patients were selected from the Department of Veterans Affairs National Surgical Quality Improvement Program (NSQIP). A detailed description of the study methods is published elsewhere and is summarized briefly here (13, 2123). Participating Hospitals Our study includes patients enrolled between 1 September 1995 and 31 August 1999 at 100 Veterans Affairs medical centers that perform major surgery. Patients enrolled at 97 medical centers between 1 September 1997 and 31 August 1999 were used to develop the model, and patients enrolled at 100 medical centers between 1 September 1995 and 31 August 1997 were used to validate the model. Selection of Patients All noncardiac operations performed under general, spinal, epidural, local, or monitored anesthesia were eligible for inclusion as index operations. All eligible operations were included at low-volume hospitals (<140 eligible operations per month). At 5 high-volume hospitals ( 140 eligible operations per month), the first 36 consecutive eligible operations were entered in each consecutive 8-day period, beginning with a different day each period. On the basis of a review of operative mortality rates from the Veterans Affairs administrative discharge database for 1988 and 1989, selected operations with very low mortality rates were excluded from the NSQIP. Major transplantation procedures and patients entered into the study within the previous 30 days were also excluded. Patients were defined as having postoperative pneumonia if they met the Centers for Disease Control and Prevention definition of nosocomial pneumonia after surgery (Table 1) (24). We excluded patients who were ventilator dependent before surgery and those who had preoperative pneumonia. Patients who experienced postoperative respiratory failure or unplanned intubation before postoperative pneumonia was diagnosed were also excluded. Table 1. Definition of Postoperative Pneumonia Data Collection A surgical risk-assessment nurse was assigned at each center to collect the data. All variables were defined before data collection (21). The nurses completed in-depth training on all study definitions and conduct of the protocol. Annual meetings and regular conference calls were conducted, and two traveling nurse coordinators performed site visits to maintain data reliability. Generic preoperative, intraoperative, and postoperative variables were chosen on the basis of clinical relevance, reliability of data collection, and availability of data. Preoperative data were obtained by the study nurse from the medical chart or a surgical risk-assessment profile that was completed by the surgical resident caring for the patient and later verified by the study nurse from the medical record. Preoperative laboratory values within 30 days of the index operation and closest to the time of operation were acquired automatically from the laboratory software in each center's computer system. Intraoperative variables were collected from the operative log and anesthesia record for the index operation. The index operation was defined as the first eligible operation performed on the patient. Other operations performed under the same anesthetic by the same or different surgical team were also recorded. The type of surgery was primarily classified by the anatomic location of the surgical incision (13). On the 30th postoperative day, the study nurse obtained outcome information by chart review, interviews with care providers, reports from morbidity and mortality conferences, and communication with each patient by letter or by telephone. Postoperative mortality was defined as death from any cause inside or outside the hospital within 30 days of surgery. The nurse entered all data into a surgical risk assessment module in each center's computer system. A summary of each patient data record was forwarded to the chief of the surgical service 30 days after surgery for inspection. No later than 45 days after surgery, the patient records were transmitted automatically to the statistical coordinating center for editing and analysis. Missing Data Data were more than 92% complete, except for preoperative laboratory variables; the completeness of these depended on whether the specific laboratory tests had been ordered. Patients with missing data for tests that were not ordered more than 8% of the time were excluded from the development and validation data sets. Twelve percent of blood urea nitrogen and 9% of creatinine values were missing; thus, patients for whom these values were missing were excluded from the development and validation data sets. Statistical Analysis Data were analyzed by using two software packages (SAS for UNIX [SAS Institute, Inc., Cary, North Carolina] and Stata statistical software, version 7.0 for Windows [Stata Corp., College Station, Texas]). Using patients from the development set (enrolled between 1 September 1997 and 31 August 1999), we used the chi-square test for categorical variables and the Student t-test for continuous variables to test the relationship between each potential risk factor and postoperative pneumonia. All potential risk factors were entered into a logistic regression model with postoperative pneumonia as the dependent variable and the potential risk factors as the independent variables. Blood urea nitrogen was categorized into ranges suggested by its U-shaped relationship to postoperative pneumonia in univariate analyses. Indicator variables created for each range were included in the logistic regression model. Potential risk factors that were not statistically significant (P < 0.05) were sequentially deleted from the full model. Excluded variables were reintroduced at various stages of model development until only statistically significant risk factors remained. Two-way interaction between recent smoking and history of chronic obstructive pulmonary disease was analyzed and found to be statistically nonsignificant. Robust variance estimates were calculated by using the HuberWhite method to account for the effect of clustering of patients within hospitals (25). Development of a Scoring System By using the methods of Le Gall and colleagues (26), point values were assigned to each risk factor by multiplying the -coefficients from the logistic regression model by 10 and rounding off to the nearest integer. The point total for each patient was designated the postoperative pneumonia risk index and was used in a multiple logistic regression equation designed to convert the risk index to the probability of developing postoperative pneumonia. Because the risk index scores were highly skewed, a shrinking power transformation, ln (postoperative pneumonia risk index score +25), where ln indicates the natural logarithm, was incorporated into the model for improved calibration (26). Evaluation of Model Performance Formal goodness-of-fit (HosmerLemeshow) tests (27) were performed by using data on patients from the development cohort, the validation cohort, and 10 random sets of five hospitals to assess model calibration. The C-statistic was used to evaluate discrimination in model development and validation (28). Results Patient Characteristics and Postoperative Pneumonia Rates Of the 206 434 patients enrolled between 1 September 1997 and 31 August 1999, 5395 patients (2.6%) with preoperative pneumonia or dependence on a ventilator were excluded. An additional 729 patients were excluded because postoperative