Magnetic-Visual Sensor Fusion based Medical SLAM for Endoscopic Capsule Robot

A reliable, real-time simultaneous localization and mapping (SLAM) method is crucial for the navigation of actively controlled capsule endoscopy robots. These robots are an emerging, minimally invasive diagnostic and therapeutic technology for use in the gastrointestinal (GI) tract. In this study, we propose a dense, non-rigidly deformable, and real-time map fusion approach for actively controlled endoscopic capsule robot applications. The method combines magnetic and vision based localization, and makes use of frame-to-model fusion and model-to-model loop closure. The performance of the method is demonstrated using an ex-vivo porcine stomach model. Across four trajectories of varying speed and complexity, and across three cameras, the root mean square localization errors range from 0.42 to 1.92 cm, and the root mean square surface reconstruction errors range from 1.23 to 2.39 cm.

[1]  Kaveh Pahlavan,et al.  Hybrid Localization of Microrobotic Endoscopic Capsule Inside Small Intestine by Data Fusion of Vision and RF Sensors , 2015, IEEE Sensors Journal.

[2]  M. Goenka,et al.  Capsule endoscopy: Present status and future expectation. , 2014, World journal of gastroenterology.

[3]  Weihua Li,et al.  A review of drug delivery systems for capsule endoscopy. , 2014, Advanced drug delivery reviews.

[4]  P. Swain,et al.  Wireless capsule endoscopy. , 2002, The Israel Medical Association journal : IMAJ.

[5]  Guang-Zhong Yang,et al.  Dynamic view expansion for minimally invasive surgery using simultaneous localization and mapping , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[6]  A. Juloski,et al.  Method for navigation and control of a magnetically guided capsule endoscope in the human stomach , 2012, 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[7]  Guang-Zhong Yang,et al.  Metric depth recovery from monocular images using Shape-from-Shading and specularities , 2012, 2012 19th IEEE International Conference on Image Processing.

[8]  Guang-Zhong Yang,et al.  Real-Time Stereo Reconstruction in Robotically Assisted Minimally Invasive Surgery , 2010, MICCAI.

[9]  Metin Sitti,et al.  Design and Rolling Locomotion of a Magnetically Actuated Soft Capsule Endoscope , 2012, IEEE Transactions on Robotics.

[10]  M. Sitti,et al.  Magnetically Actuated Soft Capsule With the Multimodal Drug Release Function , 2013, IEEE/ASME Transactions on Mechatronics.

[11]  Pietro Valdastri,et al.  Real-Time Pose Detection for Magnetic Medical Devices , 2013, IEEE Transactions on Magnetics.

[12]  Alexandre Hostettler,et al.  ORBSLAM-Based Endoscope Tracking and 3D Reconstruction , 2016, CARE@MICCAI.

[13]  Li Liu,et al.  Capsule endoscope localization based on computer vision technique , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[14]  Jake J. Abbott,et al.  An omnidirectional electromagnet for remote manipulation , 2013, 2013 IEEE International Conference on Robotics and Automation.

[15]  Manuel Davy,et al.  Particle Filtering for Multisensor Data Fusion With Switching Observation Models: Application to Land Vehicle Positioning , 2007, IEEE Transactions on Signal Processing.

[16]  Michael Talcott,et al.  Magnetically Controllable Gastrointestinal Steering of Video Capsules , 2011, IEEE Transactions on Biomedical Engineering.

[17]  Konstantinos Daniilidis.  Dual quaternions for absolute orientation and hand-eye calibration , 1996, TFCV.

[18]  Stefan Leutenegger,et al.  ElasticFusion: Real-time dense SLAM and light source estimation , 2016, Int. J. Robotics Res..

[19]  J. M. M. Montiel,et al.  Visual SLAM for Handheld Monocular Endoscope , 2014, IEEE Transactions on Medical Imaging.

[20]  Metin Sitti,et al.  A 5-D Localization Method for a Magnetically Manipulated Untethered Robot Using a 2-D Array of Hall-Effect Sensors , 2016, IEEE/ASME Transactions on Mechatronics.

[21]  Yu Sun,et al.  Simultaneous Tracking, 3D Reconstruction and Deforming Point Detection for Stereoscope Guided Surgery , 2013, AE-CAI.

[22]  Jake J. Abbott,et al.  Managing the attractive magnetic force between an untethered magnetically actuated tool and a rotating permanent magnet , 2013, 2013 IEEE International Conference on Robotics and Automation.

[23]  Kaveh Pahlavan,et al.  On the accuracy of RF and image processing based hybrid localization for wireless capsule endoscopy , 2015, 2015 IEEE Wireless Communications and Networking Conference (WCNC).

[24]  Seth D. Crockett,et al.  Burden of gastrointestinal disease in the United States: 2012 update. , 2012, Gastroenterology.

[25]  Jake J. Abbott,et al.  Localization method for a magnetic capsule endoscope propelled by a rotating magnetic dipole field , 2013, 2013 IEEE International Conference on Robotics and Automation.

[26]  Metin Sitti,et al.  3-D Localization Method for a Magnetically Actuated Soft Capsule Endoscope and Its Applications , 2013, IEEE Transactions on Robotics.

[27]  Eric Diller,et al.  Biomedical Applications of Untethered Mobile Milli/Microrobots , 2015, Proceedings of the IEEE.

[28]  Arianna Menciassi,et al.  Wireless capsule endoscopy: from diagnostic devices to multipurpose robotic systems , 2007, Biomedical microdevices.

[29]  Guang-Zhong Yang,et al.  Motion Compensated SLAM for Image Guided Surgery , 2010, MICCAI.

[30]  Weihua Li,et al.  A Review of Localization Systems for Robotic Endoscopic Capsules , 2012, IEEE Transactions on Biomedical Engineering.

[31]  Metin Sitti,et al.  Biopsy using a Magnetic Capsule Endoscope Carrying, Releasing, and Retrieving Untethered Microgrippers , 2014, IEEE Transactions on Biomedical Engineering.

[32]  Metin Sitti,et al.  Shape-Programmable Soft Capsule Robots for Semi-Implantable Drug Delivery , 2012, IEEE Transactions on Robotics.

[33]  Alejandro F. Frangi,et al.  Muliscale Vessel Enhancement Filtering , 1998, MICCAI.

[34]  Tetsuya Nakamura,et al.  Capsule endoscopy: past, present, and future , 2008, Journal of Gastroenterology.

[35]  Guang-Zhong Yang,et al.  Simultaneous Stereoscope Localization and Soft-Tissue Mapping for Minimal Invasive Surgery , 2006, MICCAI.