Artificial intelligence-driven real-time 3D surface quantification of Barrett's oesophagus for risk stratification and therapeutic response monitoring

BACKGROUND & AIMS: Barrett's epithelium measurement using widely accepted Prague C&M criteria is highly operator dependent. By reconstructing the surface of the Barrett's area in 3D from endoscopy video, we propose a novel methodology for measuring the C&M score automatically. This 3D reconstruction provides an extended field of view and also allows to precisely quantify the Barrett's area including islands. We aim to assess the accuracy of the extracted measurements from phantom and demonstrate their clinical usability. METHODS: Advanced deep learning techniques are utilised to design estimators for depth and camera pose required to map standard endoscopy video to a 3D surface model. By segmenting the Barrett's area and locating the position of the gastro-oesophageal junction (GEJ) we measure C&M scores and the Barrett's oesophagus areas (BOA). Experiments using a purpose-built 3D printed oesophagus phantom and high-definition video from 98 patients scored by an expert endoscopist are used for validation. RESULTS: Endoscopic phantom video data demonstrated a 95 % accuracy with a marginal +/- 1.8 mm average deviation for C&M and island measurements, while for BOA we achieved nearly 93 % accuracy with only +/- 1.1 sq. cm average deviation compared to the ground-truth measurements. On patient data, the C&M measurements provided by our system concord with the reference provided by expert upper GI endoscopists. CONCLUSIONS: The proposed methodology is suitable for extracting Prague C&M scores automatically with a high degree of accuracy. Providing an accurate measurement of the entire Barrett's area provides new opportunities for risk stratification and the assessment of therapy response.