The eGVHD App has the potential to improve the accuracy of graft-versus-host disease assessment: a multicenter randomized controlled trial

Graft-versus-host disease (GvHD) assessment has been shown to be a challenge for healthcare professionals, leading to the development of the eGVHD App (www.uzleuven.be/egvhd). In this study, we formally evaluated the accuracy of using the App compared to traditional assessment methods to assess GvHD. Our national multicenter randomized controlled trial involved seven Belgian transplantation centers and 78 healthcare professionals selected using a 2-stage convenience sampling approach between January and April 2017. Using a 1:1 randomization stratified by profession, healthcare professionals were assigned to use either the App (“APP”) or their usual GvHD assessment aids (“No APP”) to assess the diagnosis and severity score of 10 expert-validated clinical vignettes. Our main outcome measure was the difference in accuracy for GvHD severity scoring between both groups. The odds of being correct were 6.14 (95%CI: 2.83–13.34) and 6.29 (95%CI: 4.32–9.15) times higher in favor of the “APP” group for diagnosis and scoring, respectively (P<0.001). App-assisted GvHD severity scoring was significantly superior for both acute and chronic GvHD, with an Odds Ratio of 17.89 and 4.34 respectively (P<0.001) and showed a significantly increased inter-observer agreement compared to standard practice. Despite a mean increase of 24 minutes (95%CI: 20.45–26.97) in the time needed to score the whole GvHD test package in the “APP” group (P<0.001), usability feedback was positive. The eGVHD App shows superior GvHD assessment accuracy compared to standard practice and has the potential to improve the quality of outcome data registration in allogeneic stem cell transplantation.

[1]  B. Storer,et al.  Organ Changes Associated with Provider-Assessed Responses in Patients with Chronic Graft-versus-Host Disease. , 2019, Biology of blood and marrow transplantation : journal of the American Society for Blood and Marrow Transplantation.

[2]  M. Rubio eGVHD App: a new tool to improve graft-versus-host disease assessment , 2018, Haematologica.

[3]  E. Holler,et al.  EBMT—NIH—CIBMTR Task Force position statement on standardized terminology & guidance for graft-versus-host disease assessment , 2018, Bone Marrow Transplantation.

[4]  M. Perales,et al.  Establishing a Standardized System to Capture Chronic GRAFT-Versus-Host Disease (GVHD) DATA in Accordance to the National Institutes of Health (NIH) Consensus Criteria , 2018 .

[5]  F. Dobbels,et al.  Accuracy and usability of the eGVHD app in assessing the severity of graft-versus-host disease at the 2017 EBMT annual congress , 2018, Bone Marrow Transplantation.

[6]  K. Gwet Testing the Difference of Correlated Agreement Coefficients for Statistical Significance , 2016, Educational and psychological measurement.

[7]  F. Dobbels,et al.  Development, preliminary usability and accuracy testing of the EBMT ‘eGVHD App’ to support GvHD assessment according to NIH criteria—a proof of concept , 2016, Bone Marrow Transplantation.

[8]  M. Horowitz,et al.  Prednisone (PDN)/Sirolimus (SRL) Compared to PDN/SRL/Calcineurin Inhibitor (CNI) as Treatment for Chronic Graft-Versus-Host-Disease (cGVHD): A Randomized Phase II Study from the Blood and Marrow Transplant Clinical Trials Network , 2016 .

[9]  D. Porter,et al.  International, Multicenter Standardization of Acute Graft-versus-Host Disease Clinical Data Collection: A Report from the Mount Sinai Acute GVHD International Consortium. , 2016, Biology of blood and marrow transplantation : journal of the American Society for Blood and Marrow Transplantation.

[10]  Daniel Wolff,et al.  Measuring Therapeutic Response in Chronic Graft-versus-host-disease: National Institutes of Health Consensus Development Project on Criteria for Clinical Trials in Chronic Graft-versus- Host Disease: Iv. Response Criteria Working Group Report , 2022 .

[11]  M. Robin,et al.  A refined risk score for acute graft-versus-host disease that predicts response to initial therapy, survival, and transplant-related mortality. , 2015, Biology of blood and marrow transplantation : journal of the American Society for Blood and Marrow Transplantation.

[12]  H. Greinix,et al.  National Institutes of Health Consensus Development Project on Criteria for Clinical Trials in Chronic Graft-versus-Host Disease , 2015 .

[13]  Daniel Wolff,et al.  National Institutes of Health Consensus Development Project on Criteria for Clinical Trials in Chronic Graft-versus-Host Disease: I. The 2014 Diagnosis and Staging Working Group report. , 2005, Biology of blood and marrow transplantation : journal of the American Society for Blood and Marrow Transplantation.

[14]  Y. Efebera,et al.  Improved accuracy of acute graft-versus-host disease staging among multiple centers. , 2014, Best practice & research. Clinical haematology.

[15]  Li Li,et al.  A multicenter pilot evaluation of the National Institutes of Health chronic graft-versus-host disease (cGVHD) therapeutic response measures: feasibility, interrater reliability, and minimum detectable change. , 2011, Biology of blood and marrow transplantation : journal of the American Society for Blood and Marrow Transplantation.

[16]  E. Holler,et al.  Consensus conference on clinical practice in chronic graft-versus-host disease (GVHD): first-line and topical treatment of chronic GVHD. , 2010, Biology of blood and marrow transplantation : journal of the American Society for Blood and Marrow Transplantation.

[17]  Alexander von Eye,et al.  On the Marginal Dependency of Cohen’s κ , 2008 .

[18]  H. Deeg,et al.  How I treat refractory acute GVHD. , 2007, Blood.

[19]  M. Turner,et al.  National Institutes of Health consensus development project on criteria for clinical trials in chronic graft-versus-host disease: I. Diagnosis and staging working group report. , 2005, Biology of blood and marrow transplantation : journal of the American Society for Blood and Marrow Transplantation.

[20]  J. Wagner,et al.  Prospective grading of graft-versus-host disease after unrelated donor marrow transplantation: a grading algorithm versus blinded expert panel review. , 2003, Biology of blood and marrow transplantation : journal of the American Society for Blood and Marrow Transplantation.

[21]  K. Sullivan,et al.  A retrospective analysis of therapy for acute graft-versus-host disease: initial treatment. , 1990, Blood.

[22]  P. Neiman,et al.  CLINICAL MANIFESTATIONS OF GRAFT‐VERSUS-HOST DISEASE IN HUMAN RECIPIENTS OF MARROW FROM HL‐A-MATCHED SIBLING DONOR,S , 1974, Transplantation.