Image-Based Incision Detection for Topological Intraoperative 3D Model Update in Augmented Reality Assisted Laparoscopic Surgery

37 EMIM 2021 Abstract Jun 11, 2021, 9:36:28 AM Page 1/3 Abstract #37 | Oral or Poster presentation Image-based Incision Detection and Topological Intraoperative 3D Model Update in Augmented Reality Assisted Laparoscopic Surgery Tom François , Lilian Calvet , Callyane Sève-d’Erceville, Nicolas Bourdel, Adrien Bartoli 1 Université Clermont-Auvergne, Institut Pascal, Clermont-Ferrand, France 2 Be-Ys Research, Châtelaine, Switzerland 3 INP Toulouse, Toulouse, France Introduction Augmented Reality allows one to visualise the internal structures of an organ in laparosurgery. It requires one to register a preoperative 3D model obtained from MRI or CT to laparoscopic images. Registration is a challenging problem because of the deformations. Existing methods assume a fixed topology of the model, which leads to registration failure during organ incision. Registering and handling topological model changes forms an open problem. Solving it would extend gesture guidance to some critical parts of surgical procedures. Methods We propose ImTopUp, an image-based topological update registration framework for incision aware registration in augmented laparosurgery, described in figure 1. [1], which we named GeoTopUp, detects incisions with a geometric criterion using keypoint tracks across the laparoscopic video. It does not use dense pixel information indicating organ incision from colour change or bleeding. In contrast, ImTopUp uses image-based incision detection with UNet [4] trained from 181 incision examples from 10 uterus surgeries. Once detected, the incision is transferred to the model. The transfer uses a warp to a reference image pre-registered to the model. The model topology is then updated and registration to the input image solved.