The ADNEX risk prediction model for ovarian cancer diagnosis: A systematic review and meta-analysis of external validation studies

Objectives: To conduct a systematic review of studies externally validating the ADNEX model for ovarian cancer diagnosis and perform a meta-analysis of its performance. Design: Systematic review, meta-analysis Data sources: Medline, EMBASE, WOS, Scopus, and EuropePMC up to 15/05/2023. Review methods: We included external validation studies of the performance of ADNEX using any study design and any study population comprising patients with an adnexal mass. Two independent reviewers extracted data. Disagreements were resolved through discussion. Reporting quality of the studies was scored using the TRIPOD reporting guideline and methodological conduct and risk of bias using the PROBAST tool. We performed random effects meta-analysis of the AUC, sensitivity and specificity at the 10% risk of malignancy threshold, and Net Benefit and Relative Utility at the 10% risk of malignancy threshold. Results: We included 47 studies (17,007 tumours) with median study sample size 261 (range 24-4905). On average, 61% of TRIPOD items were reported. Handling of missing data, sample size justification, and model calibration were rarely described. 91% of validations were at high risk of bias, mainly due to the unexplained exclusion of incomplete cases, low sample size, or absent calibration assessment. The summary AUC to distinguish benign from malignant tumours in operated patients was 0.93 (95% CI 0.92-0.94, 95% prediction interval 0.85-0.98) for ADNEX with CA125 as a predictor (9202 tumours, 43 centres, 18 countries, 21 studies) and 0.93 (95% CI 0.91-0.94, 95% prediction interval 0.85-0.98) for ADNEX without CA125 (6309 tumours, 31 centres, 13 countries, 12 studies). The estimated probability that the model has clinical utility in a new centre was 95% (with CA125) and 91% (without CA125). When restricting analysis to studies at low risk of bias, summary AUCs were 0.93 (with CA125) and 0.91 (without CA125), and estimated probabilities that the model has clinical utility were 89% (with CA125) and 87% (without CA125). Discussion: ADNEX performed well to distinguish benign from malignant tumours in populations from different countries and settings regardless of whether CA125 was used or not. A key limitation is that calibration was rarely assessed. Review registration: PROSPERO, CRD42022373182

[1]  L. Hooft,et al.  Transparent reporting of multivariable prediction models for individual prognosis or diagnosis: checklist for systematic reviews and meta-analyses (TRIPOD-SRMA) , 2023, BMJ.

[2]  M. van Smeden,et al.  Imputation and missing indicators for handling missing data in the development and deployment of clinical prediction models: A simulation study , 2023, Statistical methods in medical research.

[3]  T. Lei,et al.  Comparison of performance between O-RADS, IOTA simple rules risk assessment and ADNEX model in the discrimination of ovarian Brenner tumors , 2023, Archives of Gynecology and Obstetrics.

[4]  Wei Tian,et al.  Comparison of ultrasound−based ADNEX model with magnetic resonance imaging for discriminating adnexal masses: a multi-center study , 2023, Frontiers in Oncology.

[5]  M. Sable,et al.  Diagnostic Performance of Ultrasound-Based International Ovarian Tumor Analysis Simple Rules and Assessment of Different NEoplasias in the adneXa Model for Predicting Malignancy in Women with Ovarian Tumors: A Prospective Cohort Study , 2023, Women's health reports.

[6]  J. Alcázar,et al.  Comparison of Ultrasound Scores in Differentiating between Benign and Malignant Adnexal Masses , 2023, Diagnostics.

[7]  P. Tanmahasamut,et al.  Prospective comparative trial comparing O‐RADS, IOTA ADNEX model, and RMI score for preoperative evaluation of adnexal masses for prediction of ovarian cancer , 2023, The journal of obstetrics and gynaecology research.

[8]  Zongli Yang,et al.  Evaluating the risk of malignancy in adnexal masses: validation of O-RADS and comparison with ADNEX model, SA, and RMI. , 2023, Ginekologia polska.

[9]  A. Smoleń,et al.  Performance of IOTA Simple Rules Risks, ADNEX Model, Subjective Assessment Compared to CA125 and HE4 with ROMA Algorithm in Discriminating between Benign, Borderline and Stage I Malignant Adnexal Lesions , 2023, Diagnostics.

[10]  B. van Calster,et al.  There is no such thing as a validated prediction model , 2023, BMC Medicine.

[11]  C. Velayo,et al.  Diagnostic Performances of Ultrasound-Based Models for Predicting Malignancy in Patients with Adnexal Masses , 2022, Healthcare.

[12]  T. Bourne,et al.  Benign descriptors and ADNEX in two‐step strategy to estimate risk of malignancy in ovarian tumors: retrospective validation in IOTA5 multicenter cohort , 2022, Ultrasound in obstetrics & gynecology : the official journal of the International Society of Ultrasound in Obstetrics and Gynecology.

[13]  Su-Yueh Yang,et al.  Performance of the IOTA ADNEX model combined with HE4 for identifying early-stage ovarian cancer , 2022, Frontiers in Oncology.

[14]  G. Collins,et al.  Minimum sample size for developing a multivariable prediction model using multinomial logistic regression , 2022, Statistical methods in medical research.

[15]  J. Deeks,et al.  Diagnostic Models Combining Clinical Information, Ultrasound and Biochemical Markers for Ovarian Cancer: Cochrane Systematic Review and Meta-Analysis , 2022, Cancers.

[16]  J. Deeks,et al.  Menopausal status, ultrasound and biomarker tests in combination for the diagnosis of ovarian cancer in symptomatic women. , 2022, The Cochrane database of systematic reviews.

[17]  Yen-Hou Chang,et al.  Comparison of the O-RADS and ADNEX models regarding malignancy rate and validity in evaluating adnexal lesions , 2022, European Radiology.

[18]  D. van Klaveren,et al.  Does poor methodological quality of prediction modeling studies translate to poor model performance? An illustration in traumatic brain injury , 2022, Diagnostic and Prognostic Research.

[19]  Niket Gandhi,et al.  External Validation of O-RADS US Risk Stratification and Management System. , 2022, Radiology.

[20]  I. Budiana,et al.  Skor assessment of different neoplasias in the adnexa (ADNEX) untuk memprediksi keganasan ovarium di RSUP Sanglah Denpasar , 2022, Intisari Sains Medis.

[21]  Chenyang Zhang,et al.  Value of Assessment of Different Neoplasias in the Adnexa in the Differential Diagnosis of Malignant Ovarian Tumor and Benign Ovarian Tumor: A Meta-analysis. , 2022, Ultrasound in medicine & biology.

[22]  A. Adibi,et al.  Comparison of Ultrasound and Tumor Marker CA125 in Diagnosis of Adnexal Mass Malignancies , 2022, Advanced biomedical research.

[23]  Li Feng,et al.  External Validation of the Assessment of Different NEoplasias in the adneXa Model Performance in Evaluating the Risk of Ovarian Carcinoma Before Surgery in China: A Tertiary Center Study , 2021, Journal of ultrasound in medicine : official journal of the American Institute of Ultrasound in Medicine.

[24]  Qing-qing Wu,et al.  Estimating the risk of malignancy of adnexal masses: validation of the ADNEX model in the hands of nonexpert ultrasonographers in a gynaecological oncology centre in China , 2021, Journal of Ovarian Research.

[25]  I. Vasilache,et al.  Ovarian Masses-Applicable IOTA ADNEX Model versus Morphological Findings for Accurate Diagnosis and Treatment , 2021, Applied Sciences.

[26]  G. Collins,et al.  Risk of bias in studies on prediction models developed using supervised machine learning techniques: systematic review , 2021, BMJ.

[27]  G. Lyu,et al.  Comparison of O‐RADS, GI‐RADS, and ADNEX for Diagnosis of Adnexal Masses: An External Validation Study Conducted by Junior Sonologists , 2021, Journal of ultrasound in medicine : official journal of the American Institute of Ultrasound in Medicine.

[28]  Mi-Young Lee,et al.  Ultrasonographic evaluation of ovarian mass for predicting malignancy in pregnant women. , 2021, Gynecologic oncology.

[29]  A. Hiett,et al.  Performance of IOTA Simple Rules, Simple Rules risk assessment, ADNEX model and O‐RADS in differentiating between benign and malignant adnexal lesions in North American women , 2021, Ultrasound in obstetrics & gynecology : the official journal of the International Society of Ultrasound in Obstetrics and Gynecology.

[30]  X. Peng,et al.  Evaluation of the Diagnostic Value of the Ultrasound ADNEX Model for Benign and Malignant Ovarian Tumors , 2021, International journal of general medicine.

[31]  K. Salvesen,et al.  External validation of prognostic models to predict stillbirth using International Prediction of Pregnancy Complications (IPPIC) Network database: individual participant data meta‐analysis , 2021, Ultrasound in obstetrics & gynecology : the official journal of the International Society of Ultrasound in Obstetrics and Gynecology.

[32]  Liang Wang,et al.  ADNEX Model-Based Diagnosis of Ovarian Cancer Using MRI Images , 2021, Contrast media & molecular imaging.

[33]  Indrė Tavoraitė,et al.  Ultrasound Assessment of Adnexal Pathology: Standardized Methods and Different Levels of Experience , 2021, Medicina.

[34]  Yunfeng Zhang,et al.  Diagnostic Accuracies of the Ultrasound and Magnetic Resonance Imaging ADNEX Scoring Systems For Ovarian Adnexal Mass: Systematic Review and Meta-Analysis. , 2021, Academic radiology.

[35]  G. Collins,et al.  Reporting of prognostic clinical prediction models based on machine learning methods in oncology needs to be improved , 2021, Journal of clinical epidemiology.

[36]  Hong Luo,et al.  Diagnostic Accuracy of the ADNEX Model for Ovarian Cancer at the 15% Cut-Off Value: A Systematic Review and Meta-Analysis , 2021, Frontiers in Oncology.

[37]  Hui-Ni Chen,et al.  Comparison of the Diagnostic Performances of Ultrasound-Based Models for Predicting Malignancy in Patients With Adnexal Masses , 2021, Frontiers in Oncology.

[38]  G. Collins,et al.  Minimum sample size for external validation of a clinical prediction model with a binary outcome , 2021, Statistics in medicine.

[39]  J. Alcázar,et al.  Two‐Step Strategy for Optimizing the Preoperative Classification of Adnexal Masses in a University Hospital, Using International Ovarian Tumor Analysis Models , 2021, Journal of ultrasound in medicine : official journal of the American Institute of Ultrasound in Medicine.

[40]  Hui-Ni Chen,et al.  Ovarian sex cord stromal tumours: analysis of the clinical and sonographic characteristics of different histopathologic subtypes , 2021, Journal of Ovarian Research.

[41]  T. Bourne,et al.  ESGO/ISUOG/IOTA/ESGE Consensus Statement on pre-operative diagnosis of ovarian tumors , 2021, International Journal of Gynecological Cancer.

[42]  E. Mayo-Wilson,et al.  The PRISMA 2020 statement: an updated guideline for reporting systematic reviews , 2021, BMJ.

[43]  A. Smoleń,et al.  Sonographic Assessment of Complex Ultrasound Morphology Adnexal Tumors in Pregnant Women with the Use of IOTA Simple Rules Risk and ADNEX Scoring Systems , 2021, Diagnostics.

[44]  P. Chantrapanichkul,et al.  Preoperative Evaluation of the ADNEX Model for the Prediction of the Ovarian Cancer Risk of Adnexal Masses at Siriraj Hospital , 2021, Gynecologic and Obstetric Investigation.

[45]  K. Jeong,et al.  Assessment of different NEoplasias in the adneXa model for differentiation of benign and malignant adnexal masses in Korean women , 2021, Obstetrics & gynecology science.

[46]  G. Collins,et al.  External validation of clinical prediction models: simulation-based sample size calculations were more reliable than rules-of-thumb , 2021, Journal of clinical epidemiology.

[47]  R. Nadišauskienė,et al.  Performance of the IOTA ADNEX Model on Selected Group of Patients with Borderline Ovarian Tumours , 2020, Medicina.

[48]  T. Bourne,et al.  Validation of models to diagnose ovarian cancer in patients managed surgically or conservatively: multicentre cohort study , 2020, BMJ.

[49]  A. Sayasneh,et al.  Surgery of Benign Ovarian Masses by a Gynecological Cancer Surgeon: A Cohort Study in a Tertiary Cancer Centre , 2020, Cureus.

[50]  B. K. Park,et al.  Validation of IOTA-ADNEX Model in Discriminating Characteristics of Adnexal Masses: A Comparison with Subjective Assessment , 2020, Journal of clinical medicine.

[51]  M. Sargin,et al.  Preoperative discriminating performance of the IOTA-ADNEX model and comparison with Risk of Malignancy Index: an external validation in a non-gynecologic oncology tertiary center , 2020 .

[52]  G. Heinze,et al.  Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal , 2020, BMJ.

[53]  P. Zola,et al.  The ADNEX model to triage adnexal masses: An external validation study and comparison with the IOTA two-step strategy and subjective assessment by an experienced ultrasound operator. , 2020, European journal of obstetrics, gynecology, and reproductive biology.

[54]  M. Jiang,et al.  Performance of IOTA ADNEX model in evaluating adnexal masses in a gynecological oncology center in China , 2019, Ultrasound in obstetrics & gynecology : the official journal of the International Society of Ultrasound in Obstetrics and Gynecology.

[55]  J. Gębicki,et al.  Ultrasound and Clinical Preoperative Characteristics for Discrimination Between Ovarian Metastatic Colorectal Cancer and Primary Ovarian Cancer: A Case-Control Study , 2019, Diagnostics.

[56]  M. Stukan,et al.  Performance of Selected Models for Predicting Malignancy in Ovarian Tumors in Relation to the Degree of Diagnostic Uncertainty by Subjective Assessment With Ultrasound , 2019, Journal of ultrasound in medicine : official journal of the American Institute of Ultrasound in Medicine.

[57]  Dirk Timmerman,et al.  Predictive analytics in health care: how can we know it works? , 2019, J. Am. Medical Informatics Assoc..

[58]  M. Stukan,et al.  Development and validation of a model that includes two ultrasound parameters and the plasma D-dimer level for predicting malignancy in adnexal masses: an observational study , 2019, BMC Cancer.

[59]  T. Bourne,et al.  Risk of complications in patients with conservatively managed ovarian tumours (IOTA5): a 2-year interim analysis of a multicentre, prospective, cohort study. , 2019, The Lancet. Oncology.

[60]  G. Chene,et al.  Reliability of IOTA score and ADNEX model in the screening of ovarian malignancy in postmenopausal women. , 2019, Journal of gynecology obstetrics and human reproduction.

[61]  Karel Moons,et al.  PROBAST: A Tool to Assess Risk of Bias and Applicability of Prediction Model Studies: Explanation and Elaboration , 2019, Annals of Internal Medicine.

[62]  J. Leng,et al.  Factors associated with misdiagnosis of frozen section of mucinous borderline ovarian tumor , 2018, The Journal of international medical research.

[63]  J. Kleijnen,et al.  Risk scores to guide referral decisions for people with suspected ovarian cancer in secondary care: a systematic review and cost-effectiveness analysis. , 2018, Health technology assessment.

[64]  Richard D Riley,et al.  A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes , 2018, Statistical methods in medical research.

[65]  R D Riley,et al.  Random‐effects meta‐analysis of the clinical utility of tests and prediction models , 2018, Statistics in medicine.

[66]  T. Van Gorp,et al.  Estimating risk of malignancy in adnexal masses: external validation of the ADNEX model and comparison with other frequently used ultrasound methods , 2017, Ultrasound in obstetrics & gynecology : the official journal of the International Society of Ultrasound in Obstetrics and Gynecology.

[67]  L. Sarian,et al.  Performance of the IOTA ADNEX model in preoperative discrimination of adnexal masses in a gynecological oncology center , 2017, Ultrasound in obstetrics & gynecology : the official journal of the International Society of Ultrasound in Obstetrics and Gynecology.

[68]  T. Bourne,et al.  Validation of the Performance of International Ovarian Tumor Analysis (IOTA) Methods in the Diagnosis of Early Stage Ovarian Cancer in a Non-Screening Population , 2017, Diagnostics.

[69]  Dirk Timmerman,et al.  Clinical Utility of Risk Models to Refer Patients with Adnexal Masses to Specialized Oncology Care: Multicenter External Validation Using Decision Curve Analysis , 2017, Clinical Cancer Research.

[70]  L. Hooft,et al.  A guide to systematic review and meta-analysis of prediction model performance , 2017, British Medical Journal.

[71]  Hossam M. Hammady,et al.  Rayyan—a web and mobile app for systematic reviews , 2016, Systematic Reviews.

[72]  Patryk Zywica,et al.  Response to letter to the editor concerning validation of IOTA ADNEX model , 2016, Gynecologic oncology reports.

[73]  K. Astruc,et al.  [Before surgery predictability of malignant ovarian tumors based on ADNEX model and its use in clinical practice]. , 2016, Gynecologie, obstetrique & fertilite.

[74]  A. Stachowiak,et al.  External validation of the IOTA ADNEX model performed by two independent gynecologic centers. , 2016, Gynecologic oncology.

[75]  T. Bourne,et al.  Evaluating the risk of ovarian cancer before surgery using the ADNEX model: a multicentre external validation study , 2016, British Journal of Cancer.

[76]  Ewout W Steyerberg,et al.  Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests , 2016, British Medical Journal.

[77]  B. van Calster,et al.  Subjective ultrasound assessment, the ADNEX model and ultrasound‐guided tru‐cut biopsy to differentiate disseminated primary ovarian cancer from metastatic non‐ovarian cancer , 2016, Ultrasound in obstetrics & gynecology : the official journal of the International Society of Ultrasound in Obstetrics and Gynecology.

[78]  I. J. P. Howard Meta-Analysis withR , 2015 .

[79]  Gary S Collins,et al.  Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement , 2015, BMC Medicine.

[80]  Dirk Timmerman,et al.  Evaluating the risk of ovarian cancer before surgery using the ADNEX model to differentiate between benign, borderline, early and advanced stage invasive, and secondary metastatic tumours: prospective multicentre diagnostic study , 2014, BMJ : British Medical Journal.

[81]  G. Collins,et al.  Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies: The CHARMS Checklist , 2014, PLoS medicine.

[82]  Sabine Van Huffel,et al.  Assessing calibration of multinomial risk prediction models , 2014, Statistics in medicine.

[83]  Shuang Wang,et al.  Hidden in plain sight: bias towards sick patients when sampling patients with sufficient electronic health record data for research , 2014, BMC Medical Informatics and Decision Making.

[84]  Claes Wohlin,et al.  Guidelines for snowballing in systematic literature studies and a replication in software engineering , 2014, EASE '14.

[85]  G. Collins,et al.  External validation of multivariable prediction models: a systematic review of methodological conduct and reporting , 2014, BMC Medical Research Methodology.

[86]  Martha Sajatovic,et al.  Clinical Prediction Models , 2013 .

[87]  Dirk Timmerman,et al.  Assessing the discriminative ability of risk models for more than two outcome categories , 2012, European Journal of Epidemiology.

[88]  H. Dickinson,et al.  Centralisation of services for gynaecological cancers - a Cochrane systematic review. , 2012, Gynecologic oncology.

[89]  E. Steyerberg,et al.  Reporting and Methods in Clinical Prediction Research: A Systematic Review , 2012, PLoS medicine.

[90]  Susan Mallett,et al.  QUADAS-2: A Revised Tool for the Quality Assessment of Diagnostic Accuracy Studies , 2011, Annals of Internal Medicine.

[91]  S. Baker Putting risk prediction in perspective: relative utility curves. , 2009, Journal of the National Cancer Institute.

[92]  Nancy R Cook,et al.  Using relative utility curves to evaluate risk prediction , 2009, Journal of the Royal Statistical Society. Series A,.

[93]  G. Guyatt,et al.  GRADE: an emerging consensus on rating quality of evidence and strength of recommendations , 2008, BMJ : British Medical Journal.

[94]  Richard D Riley,et al.  Beyond the Bench: Hunting Down Fugitive Literature , 2004, Environmental Health Perspectives.

[95]  E. Elkin,et al.  Decision Curve Analysis: A Novel Method for Evaluating Prediction Models , 2006, Medical decision making : an international journal of the Society for Medical Decision Making.

[96]  Johannes B Reitsma,et al.  Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. , 2005, Journal of clinical epidemiology.

[97]  S. Thompson,et al.  How should meta‐regression analyses be undertaken and interpreted? , 2002, Statistics in medicine.

[98]  G. Collins,et al.  PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies , 2019, Annals of Internal Medicine.

[99]  T. Bourne,et al.  O-RADS US Risk Stratification and Management System: A Consensus Guideline from the ACR Ovarian-Adnexal Reporting and Data System Committee. , 2019, Radiology.

[100]  Ewout W. Steyerberg,et al.  Updating for a New Setting , 2019, Statistics for Biology and Health.

[101]  Heinz Holling,et al.  Meta-Analysis of Diagnostic Accuracy with mada , 2015 .

[102]  Martyn Plummer,et al.  JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling , 2003 .