Reconstructing the Esophagus Surface from Endoscopic Image Sequences

The esophageal cancer is a highly lateral disease affecting a significant percentage of population in United States and in Western World [2],[8]. Recent technological developments provide powerful tools such as white light endoscopy, narrow-band endoscopic imaging (NBI) and auto-fluorescence imaging (AFI), which allow for advanced visualization of the malignant tissue. Current gold standard protocol for screening and surveillance of the esophageal cancer, known as Gastrointestinal (GI) endoscopy, consists of visual examination of the esophagus surface under endoscopic guidance and acquisition of biopsies from endoscopically visible lesions. The first time screening is followed by surveillance endoscopies at regular intervals in order to track the evolution of the malignant tissue. This protocol requires identification and retargeting of the examined regions in the surveillance endoscopy for potential biopsy acquisition. However, a method for accurate localization and retargeting of the examined regions under endoscopic guidance is currently not available. Creating a 3D visualization of the esophageal surface can serve the endoscopist as a roadmap and provide a significant support for retargeting during the procedure. In this work, we explore the possibility of creating a 3D patient-specific visualization of the esophagus surface from endoscopic images by performing a 3D reconstruction of the tissue surface. Reconstruction of the esophagus surface from endoscopic videos involves several challenges: the esophagus tissue presents big deformations which can greatly modify the appearance of the tissue; moreover, several factors affect the quality of the images, e.g. liquid inside the esophagus generates specularities and fast motion of the camera leads to blurred images. In this paper, we investigate the feasibility of using Monocular Simultaneous Localization and Mapping (MonoSLAM) [1] for 3D reconstruction of the esophagus surface in the presence of these challenges. We identify the necessary modifications for its in-vivo application and demonstrate that the MonoSLAM is a promising framework for 3D visualization of esophageal tissue. 3D surface reconstruction is a well-known research area in computer vision community. In the medical field, there has been an increasing interest in 3D surface reconstruction from endoscopic image sequences [4],[5], [6],[7],[12],[13],[9]. Stoyanov et al. [9] present a method for depth recovery from stereo laparoscopic images of deformable soft-tissue. Mourgues et al. [5] propose a correlation-based stereo method for surface reconstruction and organ modeling from stereo endoscopic images. Quartucci et al. [7] apply a shapefrom-shading technique to estimate the surface shape by recovering the depth information from the surface illumination. Zhou et al. [13] reconstruct the 3D structure using a Circular Generalized Cylinder (CGC) model which decomposes the reconstruction into a series of 3D circles forming a tube that models the esophagus. Recently, a number of techniques have been published which apply feature-based techniques for 3D reconstruction in endoscopy [4],[11],[12]. Wang et al. [11] track Scale Invariant Features (SIFT) [3] in the endoscopic sequence and use Adaptive Scale Kernel Consensus (ASKC), for robust motion estimation. In [12], Wu et al. also track SIFT features and use an iterative factorization method for structure estimation. Mountney et al. [4] present a technique for building a 3D map of the scene for minimally invasive endoscopic surgery while recovering the camera movement based on SLAM from a stereo endoscope. In this paper we propose a MonosSLAM based approach for 3D surface reconstruction from monocular endoscopic images. First, SIFT features [3] are detected in the first frame. Then, tracking of these features in consecutive frames is performed by an intensity based method using normalized sum-of-squared differences similarity measure [1],[6]. The MonoSLAM algorithm is applied on the tracked features in order to create a 3D map of the observed esophagus surface and localize the endoscope (6 DoFs) within this map simultaneously. The rest of the paper is organized as follows: the proposed approach and our results are presented in the Method and Experimental ResultsError! Reference source not found. Sections respectively. The current state and potential improvements for future work are discussed in the discussion Section.