Measures to assess the prognostic ability of the stratified Cox proportional hazards model

Many measures have been proposed to summarize the prognostic ability of the Cox proportional hazards (CPH) survival model, although none is universally accepted for general use. By contrast, little work has been done to summarize the prognostic ability of the stratified CPH model; such measures would be useful in analyses of individual participant data from multiple studies, data from multi-centre studies, and in single study analysis where stratification is used to avoid making assumptions of proportional hazards. We have chosen three measures developed for the unstratified CPH model (Schemper and Henderson's V , Harrell's C-index and Royston and Sauerbrei's D), adapted them for use with the stratified CPH model and demonstrated how their values can be represented over time. Although each of these measures is promising in principle, we found the measure of explained variation V very difficult to apply when data are combined from several studies with differing durations of participant follow-up. The two other measures considered, D and the C-index, were more applicable under such circumstances. We illustrate the methods using individual participant data from several prospective epidemiological studies of chronic disease outcomes.

Alicja R. Rudnicka | Mark Woodward | Yoav Ben-Shlomo | Simon G. Thompson | John B. Kostis | Eric J. Brunner | R. P. Tracy | Paul M. Ridker | Mary Cushman | Roberto Marchioli | P. M. Sweetnam | Helmut Schulte | Anne Tybjærg-Hansen | Philippe Amouyel | Jean Ferrières | Tomi-Pekka Tuomainen | John Danesh | Hugh Tunstall-Pedoe | Ian R. White | Jukka T. Salonen | Gilles Dagenais | Johann Willeit | Bo Hedblad | Bruce M. Psaty | G Davey Smith | F. Haverkate | B L De Stavola | Angela Wood | Akihiko Kitamura | Wolfgang Koenig | Hiroyasu Iso | J. Danesh | D. Levy | V. Salomaa | R. D'Agostino | G. Dagenais | E. Feskens | M. Woodward | H. Tunstall-Pedoe | P. Ridker | I. White | A. Wood | H. Hense | J. Després | A. Folsom | L. Chambless | P. Wilson | W. Kannel | B. Psaty | T. Palosuo | M. Walker | A. Rudnicka | A. Evans | D. Arveiler | F. Valagussa | G. Berglund | P. Jousilahti | B. Hedblad | J. Salonen | P. Amouyel | P. Brennan | S. Kiechl | J. Willeit | P. Ducimetiere | G. Assmann | H. Schulte | G. Lowe | U. Goldbourt | W. Koenig | G. Engström | K. Yano | T. Meade | Y. Ben-Shlomo | L. Wilhelmsen | N. Sarwar | J. Ferrières | J. Kostis | A. Rosengren | E. Brunner | S. Kaptoge | E. Angelantonio | F. Fowkes | J. Pekkanen | D. Kromhout | T. Tuomainen | S. Thompson | Amanda J Lee | M. Cushman | I. Ford | J. Curb | G. Tofler | H. Iso | S. Lewington | R. Tracy | A. Kitamura | J. Cooper | A. Tosetto | K. Bauer | J. Shepherd | P. Sweetnam | A. Tybjærg‐Hansen | B. Lamarche | G. Miller | G. Lappas | F. Haverkate | V. Rasi | D. Nagel | G. D. Smith | B. Rodríguez | P. Cremer | M. Benderly | H. Eriksson | L. Pennells | Shinichi Sato | Y. Naito | R. Marchioli | I. Juhan-vague | Göran Berglund | Gerd Assmann | Benoît Lamarche | Ian Ford | W B Kannel | Aaron R. Folsom | Annika Rosengren | Michele Robertson | J. Shepherd | Edith J. M. Feskens | Jean-Pierre Després | Gunnar Engström | Daan Kromhout | Stefan Kiechl | Alun Evans | Beatriz L. Rodriguez | Dominique Arveiler | Lisa Pennells | Bernard Cantin | Henry Eriksson | Lars Wilhelmsen | G. Lappas | Pierre Ducimetière | Angela Wilson | M. Benderly | Uri Goldbourt | F. G. R. Fowkes | F. B. Smith | V. Rasi | E. Vahtera | J. Pekkanen | Peter W.F. Wilson | F. Valagussa | Peter Cremer | Dorothea Nagel | J. D. Curb | K. Yano | H. Loewel | H. W. Hense | Tom Meade | J. A. Cooper | C. Knottenbelt | George J. Miller | Kenneth A. Bauer | Irène Juhan-Vague | Sarah Lewington | V. Palmieri | Alberto Tosetto | Nadeem Sarwar | M. Shipley | A. Bingham | Geoffrey H. Tofler | S. Kaptoge | Lloyd Chambless | J. W. G. Yarnell | M.P.M. de Maat | V. Salomaa | K. Harald | P. Jousilahti | R. D'Agostino | Daniel A Levy | Kristiina Nyyssönen | J. W. Cooper | J. A. Copper | R D Rosenberg | Shinichi Sato | Yoshihiko Naito | T. Palosuo | Annie Bingham | G. D. O. Lowe | J. L. Yeh | Paul Brennan | F. Rodeghiero | M J Shipley | E Di Angelantonio | M Walker | M. Robertson | B. Cantin | J. Yarnell | K. Harald | C. Knottenbelt | E. Vahtera | H. Loewel | V. Palmieri | J. Cooper | K. Nyyssönen | J. Yeh | B. D. Stavola | Angela Wilson | R. Rosenberg | F. Rodeghiero | M. Maat | J. Copper | A. Evans | J. Despres | F. Smith | B. Psaty | S. Sato | J. Cooper | P. Wilson | D. Levy

[1]  M. Schemper,et al.  Predictive Accuracy and Explained Variation in Cox Regression , 2000, Biometrics.

[2]  Alicja R. Rudnicka,et al.  Collaborative meta-analysis of prospective studies of plasma fibrinogen and cardiovascular disease , 2004, European journal of cardiovascular prevention and rehabilitation : official journal of the European Society of Cardiology, Working Groups on Epidemiology & Prevention and Cardiac Rehabilitation and Exercise Physiology.

[3]  Nancy R Cook,et al.  The Effect of Including C-Reactive Protein in Cardiovascular Risk Prediction Models for Women , 2006, Annals of Internal Medicine.

[4]  Patrick Royston,et al.  Construction and validation of a prognostic model across several studies, with an application in superficial bladder cancer , 2004, Statistics in medicine.

[5]  Daniel B. Mark,et al.  TUTORIAL IN BIOSTATISTICS MULTIVARIABLE PROGNOSTIC MODELS: ISSUES IN DEVELOPING MODELS, EVALUATING ASSUMPTIONS AND ADEQUACY, AND MEASURING AND REDUCING ERRORS , 1996 .

[6]  H Tunstall-Pedoe,et al.  Plasma fibrinogen level and the risk of major cardiovascular diseases and nonvascular mortality: an individual participant meta-analysis. , 2005, JAMA.

[7]  John O'Quigley,et al.  Explained randomness in proportional hazards models , 2005, Statistics in medicine.

[8]  H Tunstall-Pedoe,et al.  The Emerging Risk Factors Collaboration: analysis of individual data on lipid, inflammatory and other markers in over 1.1 million participants in 104 prospective studies of cardiovascular diseases , 2007, European Journal of Epidemiology.

[9]  S. Cessie,et al.  Ridge Estimators in Logistic Regression , 1992 .

[10]  P. Heagerty,et al.  Survival Model Predictive Accuracy and ROC Curves , 2005, Biometrics.

[11]  M. Schemper Predictive accuracy and explained variation , 2003, Statistics in medicine.

[12]  D. Cox Regression Models and Life-Tables , 1972 .

[13]  G. Brier VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY , 1950 .

[14]  Patrick Royston,et al.  A new measure of prognostic separation in survival data , 2004, Statistics in medicine.

[15]  Michael Schemper,et al.  The explained variation in proportional hazards regression , 1990 .

[16]  Nancy R. Cook,et al.  Use and Misuse of the Receiver Operating Characteristic Curve in Risk Prediction , 2007, Circulation.

[17]  M. Pencina,et al.  Overall C as a measure of discrimination in survival analysis: model specific population value and confidence interval estimation , 2004, Statistics in medicine.

[18]  R. Collins,et al.  Collaborative overview (‘Meta-Analysis’) of Prospective Observational Studies of the Associations of Usual Blood Pressure and Usual Cholesterol Levels with Common Causes of Death: Protocol for the Second Cycle of the Prospective Studies Collaboration , 1999, Journal of cardiovascular risk.

[19]  S R Lipsitz,et al.  A Global Goodness‐of‐Fit Statistic for Cox Regression Models , 1999, Biometrics.

[20]  F. Harrell,et al.  Evaluating the yield of medical tests. , 1982, JAMA.

[21]  Paolo Vineis,et al.  Consumption of vegetables and fruits and risk of breast cancer. , 2005, JAMA.

[22]  Michael J. Pencina,et al.  Comments on ‘Integrated discrimination and net reclassification improvements—Practical advice’ , 2008 .

[23]  R Simon,et al.  Measures of explained variation for survival data. , 1990, Statistics in medicine.

[24]  J Stare,et al.  Explained variation in survival analysis. , 1996, Statistics in medicine.

[25]  E Graf,et al.  Assessment and comparison of prognostic classification schemes for survival data. , 1999, Statistics in medicine.

[26]  Chaya S Moskowitz,et al.  Quantifying and comparing the accuracy of binary biomarkers when predicting a failure time outcome. , 2004, Statistics in medicine.

[27]  M. Pencina,et al.  Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond , 2008, Statistics in medicine.

[28]  G A Diamond,et al.  What price perfection? Calibration and discrimination of clinical prediction models. , 1992, Journal of clinical epidemiology.

[29]  David A. Schoenfeld,et al.  Partial residuals for the proportional hazards regression model , 1982 .

[30]  N. Cook,et al.  Development and validation of improved algorithms for the assessment of global cardiovascular risk in women: the Reynolds Risk Score. , 2007, JAMA.

[31]  F. Clavel-Chapelon,et al.  Body size and breast cancer risk: Findings from the European prospective investigation into cancer and nutrition (EPIC) , 2004, International journal of cancer.

[32]  Michael I. Mandel,et al.  Evaluating survival model performance: a graphical approach , 2005, Statistics in medicine.

[33]  Guoqing Diao,et al.  Estimation of time‐dependent area under the ROC curve for long‐term risk prediction , 2006, Statistics in medicine.

[34]  M. Gonen,et al.  Concordance probability and discriminatory power in proportional hazards regression , 2005 .

[35]  Ewout W Steyerberg,et al.  Validation and updating of predictive logistic regression models: a study on sample size and shrinkage , 2004, Statistics in medicine.

[36]  F. Berrino,et al.  Long-term weight change and breast cancer risk: the European prospective investigation into cancer and nutrition (EPIC) , 2005, British Journal of Cancer.