Validation of a Diagnostic Model to Differentiate Multiple Myeloma from Bone Metastasis

Purpose A diagnostic model to differentiate multiple myeloma (MM) from bone metastasis (BM) in patients with destructive bone lesions (MM-BM DDx) was developed to promote timely and appropriate referral of patients with MM to hematologists. External validation has never been conducted. This study aims to externally validate the performance of the MM-BM DDx model. Patients and Methods This multi-center external validation study was conducted using retrospective data of patients over 45 years old diagnosed with MM or BM at six university-affiliated hospitals in Thailand from 2016 to 2022. The MM-BM DDx development dataset, including patients from 2012 to 2015, was utilized during external validation. Diagnostic indicators for MM included in the MM-BM DDx model are serum creatinine, serum globulin, and serum alkaline phosphatase (ALP). MM and BM diagnosis was based on the documented International Classification of Diseases 10th Revision codes. Model performance was evaluated in terms of discrimination, calibration, and accuracy. Results A total of 3018 patients were included in the validation dataset (586 with MM and 2432 with BM). Clinical characteristics were similar between the validation and development datasets. The MM-BM DDx model’s predictions showed an AUC of 0.89 (95% CI, 0.87, 0.90). The predicted probabilities of MM from the model increased concordantly with the observed proportion of MM within the validation dataset. The estimated sensitivity, specificity, and LR for each odds class in the validation dataset were similar to those of the development dataset. Conclusion The discriminative ability and calibration of the MM-BM DDx model were found to be preserved during external validation. These findings provide support for the practical use of the MM-BM DDx model to assist clinicians in identifying patients with destructive bone lesions who are likely to have MM and enable them to arrange timely referrals for further evaluation by hematologists.

[1]  B. Shinkins,et al.  Development and Internal Validation of a Risk Prediction Model to Identify Myeloma Based on Routine Blood Tests: A Case-Control Study , 2023, Cancers.

[2]  Richard L. Altman,et al.  Problem-oriented documentation: design and widespread adoption of a novel toolkit in a commercial electronic health record , 2023, JAMIA open.

[3]  E. Rattarittamrong,et al.  Correction: Development of a clinical diagnostic tool to differentiate multiple myeloma from bone metastasis in patients with destructive bone lesions (MM-BM DDx) , 2022, BMC Primary Care.

[4]  M. Baykara,et al.  Differentiation of multiple myeloma and metastases with apparent diffusion coefficient map histogram analysis , 2022, Northern clinics of Istanbul.

[5]  K. Ramasamy,et al.  Clinical features and diagnosis of multiple myeloma: a population-based cohort study in primary care , 2021, BMJ Open.

[6]  He Huang,et al.  Measuring the global, regional, and national burden of multiple myeloma from 1990 to 2019 , 2021, BMC Cancer.

[7]  G. Collins,et al.  Minimum sample size for external validation of a clinical prediction model with a binary outcome , 2021, Statistics in medicine.

[8]  F. Hobbs,et al.  Clinical prediction tools to identify patients at highest risk of myeloma in primary care: a retrospective open cohort study , 2021, The British journal of general practice : the journal of the Royal College of General Practitioners.

[9]  Yu Zhao,et al.  Prognostic value of the albumin–globulin ratio and albumin–globulin score in patients with multiple myeloma , 2021, The Journal of international medical research.

[10]  Chunhong Hu,et al.  Differentiating Between Multiple Myeloma and Metastasis Subtypes of Lumbar Vertebra Lesions Using Machine Learning–Based Radiomics , 2021, Frontiers in Oncology.

[11]  Nicholas G Zaorsky,et al.  Epidemiology of bone metastases. , 2020, Bone.

[12]  G. Avvisati,et al.  Alkaline phosphatase (alp) levels in multiple myeloma and solid cancers with bone lesions: Is there any difference? , 2020, Journal of bone oncology.

[13]  M. Ferrone,et al.  Serum alkaline phosphatase is a prognostic marker in bone metastatic disease of the extremity. , 2020, Journal of orthopaedics.

[14]  E. Rattarittamrong,et al.  Development of a clinical diagnostic tool to differentiate multiple myeloma from bone metastasis in patients with destructive bone lesions (MM-BM DDx) , 2020, BMC Family Practice.

[15]  P. Levay,et al.  Globulin fraction and albumin: globulin ratio as a predictor of mortality in a South African multiple myeloma cohort , 2020, International journal of hematologic oncology.

[16]  M. Intorcia,et al.  Bone complications in patients with multiple myeloma in five European countries: a retrospective patient chart review , 2020, BMC Cancer.

[17]  M. Ferrone,et al.  Prognostic value of serum alkaline phosphatase in spinal metastatic disease , 2019, British Journal of Cancer.

[18]  C. Koshiaris.  Methods for reducing delays in the diagnosis of multiple myeloma , 2019, International journal of hematologic oncology.

[19]  W. Jee,et al.  Differentiation of multiple myeloma and metastases: Use of axial diffusion-weighted MR imaging in addition to standard MR imaging at 3T , 2018, PloS one.

[20]  K. Miller,et al.  Management of Metastatic Bone Disease Algorithms for Diagnostics and Treatment. , 2016, Anticancer research.

[21]  Karel G M Moons,et al.  A new framework to enhance the interpretation of external validation studies of clinical prediction models. , 2015, Journal of clinical epidemiology.

[22]  Gary S Collins,et al.  Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement , 2015, BMC Medicine.

[23]  Hans Erik Johnsen,et al.  International Myeloma Working Group updated criteria for the diagnosis of multiple myeloma. , 2014, The Lancet. Oncology.

[24]  Kihyun Kim,et al.  Renal Insufficiency in newly-diagnosed multiple myeloma: analysis according to International Myeloma Working Group consensus statement. , 2014, Anticancer research.

[25]  Patrick Royston,et al.  Tuning multiple imputation by predictive mean matching and local residual draws , 2014, BMC Medical Research Methodology.

[26]  G. Schmidt,et al.  Surgical therapy of skeletal complications in multiple myeloma , 2011, International Orthopaedics.

[27]  Yvonne Vergouwe,et al.  External validity of risk models: Use of benchmark values to disentangle a case-mix effect from incorrect coefficients. , 2010, American journal of epidemiology.

[28]  K. Weber.  Evaluation of the Adult Patient (Aged >40 Years) With a Destructive Bone Lesion , 2010, The Journal of the American Academy of Orthopaedic Surgeons.

[29]  A. Mehta,et al.  Multiple myeloma: causes and consequences of delay in diagnosis. , 2007, QJM : monthly journal of the Association of Physicians.

[30]  T. Economopoulos,et al.  Renal failure in multiple myeloma: Incidence, correlations, and prognostic significance , 2007, Leukemia & lymphoma.

[31]  J. Crowley,et al.  International staging system for multiple myeloma. , 2005, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[32]  Steven McGee,et al.  Simplifying likelihood ratios , 2002, Journal of General Internal Medicine.

[33]  Wolzt,et al.  World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. , 2003, The Journal of the American College of Dentists.

[34]  W. Jee,et al.  MR Distinction between Multiple Myeloma and Metastasis Involving the Spine , 2001 .

[35]  R. Soutar,et al.  Renal failure and multiple myeloma: pathogenesis and treatment of renal failure and management of underlying myeloma. , 1999, Blood reviews.

[36]  K. Ryu,et al.  Spinal involvement of hematopoietic malignancies and metastasis: differentiation using MR imaging. , 1999, Clinical imaging.

[37]  Douglas G Altman,et al.  Combining estimates of interest in prognostic modelling studies after multiple imputation: current practice and guidelines , 2009, BMC medical research methodology.

[38]  T. Ichida,et al.  World Medical Association declaration of Helsinki , 2008, Gastroenterologia Japonica.