A non-rigid map fusion-based direct SLAM method for endoscopic capsule robots

Abstract Since the development of capsule endoscopy technology, medical device companies and research groups have made significant progress to turn passive capsule endoscopes into robotic active capsule endoscopes. However, the use of robotic capsules in endoscopy still has some challenges. One such challenge is the precise localization of the actively controlled robot in real-time. In this paper, we propose a non-rigid map fusion based direct simultaneous localization and mapping method for endoscopic capsule robots. The proposed method achieves high accuracy for extensive evaluations of pose estimation and map reconstruction performed on a non-rigid, realistic surgical EsophagoGastroDuodenoscopy Simulator and outperforms state-of-the art methods.

[1]  M. Pauly,et al.  Embedded deformation for shape manipulation , 2007, SIGGRAPH 2007.

[2]  Paul H. J. Kelly,et al.  Dense planar SLAM , 2014, 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[3]  Mubarak Shah,et al.  Shape from shading using linear approximation , 1994, Image Vis. Comput..

[4]  Guang-Zhong Yang,et al.  Dynamic view expansion for minimally invasive surgery using simultaneous localization and mapping , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[5]  A. Juloski,et al.  Method for navigation and control of a magnetically guided capsule endoscope in the human stomach , 2012, 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[6]  John J. Leonard,et al.  Real-time large-scale dense RGB-D SLAM with volumetric fusion , 2014, Int. J. Robotics Res..

[7]  J. M. M. Montiel,et al.  Visual SLAM for Handheld Monocular Endoscope , 2014, IEEE Transactions on Medical Imaging.

[8]  M. O’Donnell,et al.  Sonographic elasticity imaging of acute and chronic deep venous thrombosis in humans. , 2007, Journal of ultrasound in medicine : official journal of the American Institute of Ultrasound in Medicine.

[9]  Yu Sun,et al.  Simultaneous Tracking, 3D Reconstruction and Deforming Point Detection for Stereoscope Guided Surgery , 2013, AE-CAI.

[10]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[11]  Stefan Leutenegger,et al.  ElasticFusion: Dense SLAM Without A Pose Graph , 2015, Robotics: Science and Systems.

[12]  Guang-Zhong Yang,et al.  Motion Compensated SLAM for Image Guided Surgery , 2010, MICCAI.

[13]  Michael Talcott,et al.  Magnetically Controllable Gastrointestinal Steering of Video Capsules , 2011, IEEE Transactions on Biomedical Engineering.

[14]  Andrew J. Davison,et al.  DTAM: Dense tracking and mapping in real-time , 2011, 2011 International Conference on Computer Vision.

[15]  Stefan Leutenegger,et al.  ElasticFusion: Real-time dense SLAM and light source estimation , 2016, Int. J. Robotics Res..

[16]  Jiawen Chen,et al.  Scalable real-time volumetric surface reconstruction , 2013, ACM Trans. Graph..

[17]  M. Fluckiger,et al.  Ultrasound Emitter Localization in Heterogeneous Media , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[18]  Metin Sitti,et al.  A 5-D Localization Method for a Magnetically Manipulated Untethered Robot Using a 2-D Array of Hall-Effect Sensors , 2016, IEEE/ASME Transactions on Mechatronics.

[19]  Eric Diller,et al.  Biomedical Applications of Untethered Mobile Milli/Microrobots , 2015, Proceedings of the IEEE.

[20]  Jake J. Abbott,et al.  An omnidirectional electromagnet for remote manipulation , 2013, 2013 IEEE International Conference on Robotics and Automation.

[21]  C. Jia,et al.  Noninvasive ultrasound elasticity imaging (UEI) of Crohn's disease: animal model. , 2008, Ultrasound in medicine & biology.

[22]  Tim Weyrich,et al.  Real-Time 3D Reconstruction in Dynamic Scenes Using Point-Based Fusion , 2013, 2013 International Conference on 3D Vision.

[23]  Tetsuya Nakamura,et al.  Capsule endoscopy: past, present, and future , 2008, Journal of Gastroenterology.

[24]  Dieter Fox,et al.  Patch Volumes: Segmentation-Based Consistent Mapping with RGB-D Cameras , 2013, 2013 International Conference on 3D Vision.

[25]  Alexandre Hostettler,et al.  ORBSLAM-Based Endoscope Tracking and 3D Reconstruction , 2016, CARE@MICCAI.

[26]  Guang-Zhong Yang,et al.  Simultaneous Stereoscope Localization and Soft-Tissue Mapping for Minimal Invasive Surgery , 2006, MICCAI.

[27]  G. Pan,et al.  Swallowable Wireless Capsule Endoscopy: Progress and Technical Challenges , 2011, Gastroenterology research and practice.

[28]  J. M. M. Montiel,et al.  ORB-SLAM: A Versatile and Accurate Monocular SLAM System , 2015, IEEE Transactions on Robotics.

[29]  Weihua Li,et al.  A Review of Localization Systems for Robotic Endoscopic Capsules , 2012, IEEE Transactions on Biomedical Engineering.

[30]  Jake J. Abbott,et al.  Managing the attractive magnetic force between an untethered magnetically actuated tool and a rotating permanent magnet , 2013, 2013 IEEE International Conference on Robotics and Automation.

[31]  Weihua Li,et al.  A review of drug delivery systems for capsule endoscopy. , 2014, Advanced drug delivery reviews.

[32]  Metin Sitti,et al.  3-D Localization Method for a Magnetically Actuated Soft Capsule Endoscope and Its Applications , 2013, IEEE Transactions on Robotics.

[33]  Metin Sitti,et al.  Biopsy using a Magnetic Capsule Endoscope Carrying, Releasing, and Retrieving Untethered Microgrippers , 2014, IEEE Transactions on Biomedical Engineering.

[34]  Wolfram Burgard,et al.  A benchmark for the evaluation of RGB-D SLAM systems , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[35]  Metin Sitti,et al.  Shape-Programmable Soft Capsule Robots for Semi-Implantable Drug Delivery , 2012, IEEE Transactions on Robotics.

[36]  Daniel Cremers,et al.  LSD-SLAM: Large-Scale Direct Monocular SLAM , 2014, ECCV.

[37]  Guang-Zhong Yang,et al.  Real-Time Stereo Reconstruction in Robotically Assisted Minimally Invasive Surgery , 2010, MICCAI.

[38]  Z. Liao,et al.  Indications and detection, completion, and retention rates of small-bowel capsule endoscopy: a systematic review. , 2010, Gastrointestinal endoscopy.

[39]  M. Goenka,et al.  Capsule endoscopy: Present status and future expectation. , 2014, World journal of gastroenterology.

[40]  Y. Istefanopulos,et al.  IEEE Engineering in Medicine and Biology Society , 2019, IEEE Transactions on Biomedical Engineering.

[41]  Metin Sitti,et al.  Design and Rolling Locomotion of a Magnetically Actuated Soft Capsule Endoscope , 2012, IEEE Transactions on Robotics.

[42]  M. Sitti,et al.  Magnetically Actuated Soft Capsule With the Multimodal Drug Release Function , 2013, IEEE/ASME Transactions on Mechatronics.