Doug Altman: Driving critical appraisal and improvements in the quality of methodological and medical research

Doug Altman was a visionary leader and one of the most influential medical statisticians of the last 40 years. Based on a presentation in the “Invited session in memory of Doug Altman” at the 40th Annual Conference of the International Society for Clinical Biostatistics (ISCB) in Leuven, Belgium and our long‐standing collaborations with Doug, we discuss his contributions to regression modeling, reporting, prognosis research, as well as some more general issues while acknowledging that we cannot cover the whole spectrum of Doug's considerable methodological output. His statement “To maximize the benefit to society, you need to not just do research but do it well” should be a driver for all researchers. To improve current and future research, we aim to summarize Doug's messages for these three topics.

[1]  S. Cessie,et al.  Hidden analyses: a review of reporting practice and recommendations for more transparent reporting of initial data analyses , 2020, BMC Medical Research Methodology.

[2]  M. Woodward,et al.  Design choices for observational studies of the effect of exposure on disease incidence , 2019, BMJ Open.

[3]  D. Rennie.  A Tribute to Douglas G. Altman, DSc. , 2019, JAMA.

[4]  Gary S. Collins,et al.  Reporting of artificial intelligence prediction models , 2019, The Lancet.

[5]  G. Collins,et al.  Uniformity in measuring adherence to reporting guidelines: the example of TRIPOD for assessing completeness of reporting of prediction model studies , 2019, BMJ Open.

[6]  Richard D Riley,et al.  A guide to systematic review and meta-analysis of prognostic factor studies , 2019, British Medical Journal.

[7]  Douglas G Altman,et al.  Statistical methodology for constructing gestational age‐related charts using cross‐sectional and longitudinal data: The INTERGROWTH‐21st project as a case study , 2018, Statistics in medicine.

[8]  R. Porcher,et al.  Overinterpretation and misreporting of prognostic factor studies in oncology: a systematic review , 2018, British Journal of Cancer.

[9]  D. Moher,et al.  Doug Altman's legacy to Cochrane and evidence synthesis. , 2018, The Cochrane database of systematic reviews.

[10]  G. Collins,et al.  Poor reporting of multivariable prediction model studies: towards a targeted implementation strategy of the TRIPOD statement , 2018, BMC Medicine.

[11]  Maarten van Smeden,et al.  Sample size for binary logistic prediction models: Beyond events per variable criteria , 2018, Statistical methods in medical research.

[12]  G. Watts.  Douglas Graham Altman , 2018, The Lancet.

[13]  I. Chalmers,et al.  Douglas G Altman: statistician, researcher, and driving force behind global initiatives to improve the reliability of health research , 2018, British medical journal.

[14]  W. Sauerbrei,et al.  Reporting Recommendations for Tumor Marker Prognostic Studies (REMARK): An Abridged Explanation and Elaboration , 2018, Journal of the National Cancer Institute.

[15]  David Moher,et al.  Flaws in the application and interpretation of statistical analyses in systematic reviews of therapeutic interventions were common: a cross-sectional analysis. , 2018, Journal of clinical epidemiology.

[16]  D. Altman,et al.  Population attributable fraction , 2018, British Medical Journal.

[17]  D. Altman,et al.  The INTERGROWTH‐21st fetal growth standards: toward the global integration of pregnancy and pediatric care , 2018, American journal of obstetrics and gynecology.

[18]  Georg Heinze,et al.  Variable selection – A review and recommendations for the practicing statistician , 2018, Biometrical journal. Biometrische Zeitschrift.

[19]  S. Lewis,et al.  Guidelines for the Content of Statistical Analysis Plans in Clinical Trials , 2017, JAMA.

[20]  Jane M Blazeby,et al.  Core Outcome Set-STAndards for Development: The COS-STAD recommendations , 2017, PLoS medicine.

[21]  Angus G K McNair,et al.  The COMET Handbook: version 1.0 , 2017, Trials.

[22]  W. Sauerbrei,et al.  Did the reporting of prognostic studies of tumour markers improve since the introduction of REMARK guideline? A comparison of reporting in published articles , 2017, PloS one.

[23]  L. Hooft,et al.  A guide to systematic review and meta-analysis of prediction model performance , 2017, British Medical Journal.

[24]  Douglas G. Altman,et al.  No rationale for 1 variable per 10 events criterion for binary logistic regression analysis , 2016, BMC Medical Research Methodology.

[25]  Douglas G. Altman,et al.  Adequate sample size for developing prediction models is not simply related to events per variable , 2016, Journal of clinical epidemiology.

[26]  L. Watering.  Faculty of 1000 evaluation for Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. , 2016 .

[27]  David Moher,et al.  A new classification of spin in systematic reviews and meta-analyses was developed and ranked according to the severity. , 2016, Journal of clinical epidemiology.

[28]  S. Goodman,et al.  Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations , 2016, European Journal of Epidemiology.

[29]  W. Sauerbrei,et al.  Improving the Prognostic Ability through Better Use of Standard Clinical Data - The Nottingham Prognostic Index as an Example , 2016, PloS one.

[30]  Gary S Collins,et al.  Sample size considerations for the external validation of a multivariable prognostic model: a resampling study , 2015, Statistics in medicine.

[31]  Gary S Collins,et al.  Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): Explanation and Elaboration , 2015, Annals of Internal Medicine.

[32]  G. Collins,et al.  Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement , 2015, BMC medicine.

[33]  Philip M. Jones,et al.  Guidelines for Reporting Health Research: A User’s Manual , 2014, Canadian Journal of Anesthesia/Journal canadien d'anesthésie.

[34]  Richard Van Noorden,et al.  The top 100 papers , 2014, Nature.

[35]  José Villar,et al.  International standards for newborn weight, length, and head circumference by gestational age and sex: the Newborn Cross-Sectional Study of the INTERGROWTH-21st Project , 2014, The Lancet.

[36]  W. Sauerbrei,et al.  REMARK (REporting Recommendations for Tumor MARKer Prognostic Studies) , 2014 .

[37]  W. Sauerbrei,et al.  STRengthening Analytical Thinking for Observational Studies: the STRATOS initiative , 2014, Statistics in medicine.

[38]  Douglas G. Altman,et al.  Improving the Transparency of Prognosis Research: The Role of Reporting, Data Sharing, Registration, and Protocols , 2014, PLoS medicine.

[39]  Douglas G Altman,et al.  The time has come to register diagnostic and prognostic research. , 2014, Clinical chemistry.

[40]  G. Collins,et al.  External validation of multivariable prediction models: a systematic review of methodological conduct and reporting , 2014, BMC Medical Research Methodology.

[41]  R. Horton,et al.  How should medical science change? , 2014, The Lancet.

[42]  David Moher,et al.  Reducing waste from incomplete or unusable reports of biomedical research , 2014, The Lancet.

[43]  P. Royston,et al.  External validation of a Cox prognostic model: principles and methods , 2013, BMC Medical Research Methodology.

[44]  Richard D Riley,et al.  Prognosis research strategy (PROGRESS) 4: Stratified medicine research , 2013, BMJ : British Medical Journal.

[45]  Richard D Riley,et al.  Prognosis research strategy (PROGRESS) 1: A framework for researching clinical outcomes , 2013, BMJ : British Medical Journal.

[46]  E. Steyerberg,et al.  Prognosis Research Strategy (PROGRESS) 2: Prognostic Factor Research , 2013, PLoS medicine.

[47]  E. Steyerberg,et al.  Prognosis Research Strategy (PROGRESS) 3: Prognostic Model Research , 2013, PLoS medicine.

[48]  Douglas G. Altman,et al.  Reporting of Noninferiority and Equivalence Randomized Trials , 2013 .

[49]  Douglas G Altman,et al.  Reporting recommendations for tumor marker prognostic studies (REMARK): explanation and elaboration , 2012, BMC Medicine.

[50]  E. Steyerberg,et al.  Reporting and Methods in Clinical Prediction Research: A Systematic Review , 2012, PLoS medicine.

[51]  Gary S Collins,et al.  Predicting risk of osteoporotic and hip fracture in the United Kingdom: prospective independent and external validation of QFractureScores , 2011, BMJ : British Medical Journal.

[52]  D. Ransohoff,et al.  Biomarker studies: a call for a comprehensive biomarker study registry , 2011, Nature Reviews Clinical Oncology.

[53]  Douglas G Altman,et al.  Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. , 2010, JAMA.

[54]  D. Moher,et al.  Transparent and accurate reporting increases reliability, utility, and impact of your research: reporting guidelines and the EQUATOR Network , 2010, BMC medicine.

[55]  Patrick Royston,et al.  Reporting methods in studies developing prognostic models in cancer: a review , 2010, BMC medicine.

[56]  Patrick Royston,et al.  Reporting performance of prognostic models in cancer: a review , 2010, BMC medicine.

[57]  D. Moher,et al.  Guidance for Developers of Health Research Reporting Guidelines , 2010, PLoS medicine.

[58]  W. Sauerbrei,et al.  Reporting of prognostic studies of tumour markers: a review of published articles in relation to REMARK guidelines , 2009, British Journal of Cancer.

[59]  K. Jordan,et al.  Improving the quality of reporting of research studies. , 2009, Musculoskeletal care.

[60]  D. Moher,et al.  Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement , 2009, BMJ : British Medical Journal.

[61]  D G Altman,et al.  Prognostic markers in cancer: the evolution of evidence from single studies to meta-analysis, and beyond , 2009, British Journal of Cancer.

[62]  Douglas G Altman,et al.  Prognostic Models: A Methodological Framework and Review of Models for Breast Cancer , 2009, Cancer investigation.

[63]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. , 2009, Journal of clinical epidemiology.

[64]  D. Moher,et al.  Improving the reporting of pragmatic trials: an extension of the CONSORT statement , 2008, BMJ : British Medical Journal.

[65]  P. Bossuyt,et al.  Reporting bias in diagnostic and prognostic studies: time for action. , 2008, Clinical chemistry.

[66]  S. Pocock,et al.  The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. , 2007, Preventive medicine.

[67]  John P A Ioannidis,et al.  Almost all articles on cancer prognostic markers report statistically significant results. , 2007, European journal of cancer.

[68]  Matthias Egger,et al.  The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies , 2007, PLoS medicine.

[69]  Richard D Riley,et al.  Prognosis research: toward evidence-based results and a Cochrane methods group. , 2007, Journal of clinical epidemiology.

[70]  A. Harris,et al.  Microvessel density as a prognostic factor in non-small-cell lung carcinoma: a meta-analysis of individual patient data. , 2007, The Lancet. Oncology.

[71]  Douglas G. Altman,et al.  Statistical reviewing policies of medical journals , 1998, Journal of General Internal Medicine.

[72]  A. Harris,et al.  Systematic Review of Multiple Studies of Prognosis: The Feasibility of Obtaining Individual Patient Data , 2007 .

[73]  Richard D Riley,et al.  Evidence-Based Assessment and Application of Prognostic Markers: The Long Way from Single Studies to Meta-Analysis , 2006 .

[74]  Patrick Royston,et al.  The cost of dichotomising continuous variables , 2006, BMJ : British Medical Journal.

[75]  Douglas G Altman,et al.  Dichotomizing continuous predictors in multiple regression: a bad idea , 2006, Statistics in medicine.

[76]  Douglas G Altman,et al.  Reporting of noninferiority and equivalence randomized trials: an extension of the CONSORT statement. , 2006, JAMA.

[77]  Richard D Riley,et al.  Primer: an evidence-based approach to prognostic markers , 2005, Nature Clinical Practice Oncology.

[78]  W. Sauerbrei,et al.  Reporting recommendations for tumor marker prognostic studies (REMARK). , 2005, Journal of the National Cancer Institute.

[79]  W. Sauerbrei,et al.  Identification of clinically useful cancer prognostic factors: what are we missing? , 2005, Journal of the National Cancer Institute.

[80]  John P A Ioannidis,et al.  Selective reporting biases in cancer prognostic factor studies. , 2005, Journal of the National Cancer Institute.

[81]  J. Ioannidis,et al.  Better Reporting of Harms in Randomized Trials: An Extension of the CONSORT Statement , 2004, Annals of Internal Medicine.

[82]  D G Altman,et al.  Missing covariate data within cancer prognostic studies: a review of current reporting and proposed guidelines , 2004, British Journal of Cancer.

[83]  D. Altman,et al.  CONSORT statement: extension to cluster randomised trials , 2004, BMJ : British Medical Journal.

[84]  Douglas G. Altman,et al.  Methodological challenges in the evaluation of prognostic factors in breast cancer , 2004, Breast Cancer Research and Treatment.

[85]  Douglas G Altman,et al.  Systematic reviews of evaluations of prognostic variables , 2001, BMJ : British Medical Journal.

[86]  D. Moher,et al.  The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials , 2001, The Lancet.

[87]  D G Altman,et al.  Statistics in medical journals: some recent trends. , 2000, Statistics in medicine.

[88]  F. Waldman,et al.  Reproducibility of p53 Immunohistochemistry in Bladder Tumors , 2000 .

[89]  D G Altman,et al.  What do we mean by validating a prognostic model? , 2000, Statistics in medicine.

[90]  J. Going,et al.  Predicting the future: a critical appraisal of cancer prognosis studies , 1999, Histopathology.

[91]  D. Altman,et al.  Measuring agreement in method comparison studies , 1999, Statistical methods in medical research.

[92]  P. Royston,et al.  Building multivariable prognostic and diagnostic models: transformation of the predictors by using fractional polynomials , 1999 .

[93]  D. Altman,et al.  Statistical reviewing for medical journals. , 1998, Statistics in medicine.

[94]  Douglas G. Altman,et al.  Approximating statistical functions by using fractional polynomial regression , 1997 .

[95]  D. Altman,et al.  Statistics notes: Cronbach's alpha , 1997 .

[96]  R. Bast,et al.  Tumor marker utility grading system: a framework to evaluate clinical utility of tumor markers. , 1996, Journal of the National Cancer Institute.

[97]  J. Wyatt,et al.  Commentary: Prognostic models: clinically useful or quickly forgotten? , 1995 .

[98]  D. Altman,et al.  Multiple significance tests: the Bonferroni method. , 1995, BMJ.

[99]  D G Altman,et al.  Comparing two methods of clinical measurement: a personal history. , 1995, International journal of epidemiology.

[100]  Curtis L. Meinert,et al.  A proposal for structured reporting of randomized controlled trials. The Standards of Reporting Trials Group. , 1994, JAMA.

[101]  P. Royston,et al.  Regression using fractional polynomials of continuous covariates: parsimonious parametric modelling. , 1994 .

[102]  D. Altman,et al.  Statistics Notes: Diagnostic tests 1: sensitivity and specificity , 1994, BMJ.

[103]  R. Persaud.  Correlation, regression, and repeated data , 1994, BMJ.

[104]  W. Sauerbrei,et al.  Dangers of using "optimal" cutpoints in the evaluation of prognostic factors. , 1994, Journal of the National Cancer Institute.

[105]  D. G. Altman,et al.  Statistical aspects of prognostic factor studies in oncology. , 1994, British Journal of Cancer.

[106]  Douglas G. Altman,et al.  Statistics Notes: Correlation, regression, and repeated data , 1994 .

[107]  D G Altman,et al.  The scandal of poor medical research , 1994, BMJ.

[108]  D. Altman,et al.  Charts of fetal size: 1. Methodology , 1994, British journal of obstetrics and gynaecology.

[109]  D. Altman.  Construction of age-related reference centiles using absolute residuals. , 1993, Statistics in medicine.

[110]  Berthold Lausen,et al.  Maximally selected rank statistics , 1992 .

[111]  M Schumacher,et al.  A bootstrap resampling procedure for model building: application to the Cox regression model. , 1992, Statistics in medicine.

[112]  D G Altman,et al.  Statistics in medical journals: developments in the 1980s. , 1991, Statistics in medicine.

[113]  D. Altman.  Categorising continuous variables. , 1991, British Journal of Cancer.

[114]  Douglas G. Altman,et al.  Practical statistics for medical research , 1990 .

[115]  M. Fernö,et al.  Flow cytometry in primary breast cancer: improving the prognostic value of the fraction of cells in the S-phase by optimal categorisation of cut-off levels. , 1990, British Journal of Cancer.

[116]  Douglas G. Altman,et al.  Statistics with confidence: Confidence intervals and statistical guidelines . , 1990 .

[117]  D. Altman,et al.  Bootstrap investigation of the stability of a Cox regression model. , 1989, Statistics in medicine.

[118]  S. Pocock,et al.  Statistical problems in the reporting of clinical trials. A survey of three medical journals. , 1987, The New England journal of medicine.

[119]  D. Altman,et al.  STATISTICAL METHODS FOR ASSESSING AGREEMENT BETWEEN TWO METHODS OF CLINICAL MEASUREMENT , 1986, The Lancet.

[120]  D. Altman,et al.  Beneficial effect of azathioprine and prediction of prognosis in primary biliary cirrhosis. Final results of an international trial. , 1985, Gastroenterology.

[121]  C. C. Chen,et al.  The bootstrap and identification of prognostic factors via Cox's proportional hazards regression model. , 1985, Statistics in medicine.

[122]  Douglas G. Altman,et al.  Measurement in Medicine: The Analysis of Method Comparison Studies , 1983 .

[123]  M. Gardner,et al.  Statistical guidelines for contributors to medical journals. , 1983, British medical journal.

[124]  D. Siegmund,et al.  Maximally Selected Chi Square Statistics , 1982 .

[125]  R. Swindell,et al.  Statistics in Practice , 1982, British Journal of Cancer.

[126]  D G Altman,et al.  Statistics in medical journals. , 1982, Statistics in medicine.

[127]  D. Altman,et al.  ENTERIC DISEASE IN SAN FRANCISCO , 1977, The Lancet.

[128]  W. Garrard,et al.  PATTERNS OF SEXUALLY TRANSMITTED ENTERIC DISEASES IN A CITY , 1977, The Lancet.

[129]  B. Clarke,et al.  LONG-TERM COMPARATIVE TRIAL OF GLIBENCLAMIDE AND CHLORPROPAMIDE IN DIET-FAILED, MATURITY-ONSET DIABETICS , 1975, The Lancet.

[130]  Douglas G. Altman,et al.  Explanation and Elaboration , 2022 .