Individual Participant Data (IPD) Meta-analyses of Diagnostic and Prognostic Modeling Studies: Guidance on Their Use

A fundamental part of medical research is the development and validation of diagnostic and prognostic prediction models [1,2]. These prediction models aim to predict the absolute probability that a certain disease or condition is currently present (diagnostic models) or that an outcome will occur within a specific follow-up period (prognostic models) for an individual subject. Prediction models typically rely on multiple predictors, which can include demographic characteristics, medical history and physical examination items, or more complex measurements from, for example, medical imaging, electrophysiology, pathology, and biomarkers. Also for diagnostic models, estimates of probabilities are rarely based on a single test, and doctors naturally integrate several patient characteristics and symptoms [3]. A broad range of prediction modeling techniques exist, like regression approaches, neural network models, decision tree models, genetic programming models, and support vector machine learning models, although prediction models developed by a multivariable regression approach are by far prevailing. It is widely recommended that a developed prediction model should not be used in practice before being externally validated—at least once—in other individuals than those used for model development [4–7]. Unfortunately, most prediction models are poorly or not at all validated, rendering interpretation of their generalizability difficult. In addition, many systematic reviews showed that for the same outcome or same target population, numerous competing models exist [8–10]. Generally speaking, researchers often ignore existing prediction models and develop yet another prediction model from their own data [2]. This practice sustains a cycle of underpowered prediction model development studies and poor knowledge about the generalizability and applicability of developed prediction models. Evidence synthesis and meta-analysis of individual participant data (IPD) from multiple studies seems to be a unique opportunity to address these problems, as it allows researchers to develop and directly validate models on large datasets and across a wide range of populations and settings, to directly test a model’s generalizability (Fig 1) [11–13]. Fig 1 Trends in publications of IPD-MA studies focusing on the development and/or validation of diagnostic or prognostic prediction models. There is currently little guidance on how to conduct an IPD meta-analysis (IPD-MA) for developing and/or validating diagnostic or prognostic prediction models [15]. To date, most IPD-MA articles focus on estimating relative quantities, like a risk ratio, hazard ratio, or odds ratio for a specific treatment or a specific etiologic factor. In contrast, prediction modeling research is focused on developing and validating multivariable models aimed at calculating an absolute risk estimate of the combined variables, rather than estimating the relative effect of a specific treatment or etiologic factor. Furthermore, prediction modeling studies focus entirely on the role and joint contribution of multiple covariates, whereas intervention studies in principle rely on randomization to reduce the role of covariates (Table 1). Hence, IPD-MAs of randomized intervention and etiological studies, which are beyond the scope of this paper and are instead addressed in the accompanying paper [16], differ from IPD-MAs of multivariable prediction models, which are the focus of this paper. Table 1 The main differences between IPD-MA of treatment intervention studies and of multivariable prediction modeling studies. We provide an overview of the advantages and limitations of IPD-MAs aiming to develop a novel prediction model or to validate one or more existing models across multiple datasets. This overview is based on published guidelines and existing recommendations for the conduct of prediction modeling studies and of IPD-MA research. We illustrate this overview with examples of recently published IPD-MAs of prediction models across various medical domains. Our aim is to help researchers, readers, reviewers, and editors to identify and understand the key issues involved with such IPD-MA projects.