A Deep Learning Based 6 Degree-of-Freedom Localization Method for Endoscopic Capsule Robots

We present a robust deep learning based 6 degrees-of-freedom (DoF) localization system for endoscopic capsule robots. Our system mainly focuses on localization of endoscopic capsule robots inside the GI tract using only visual information captured by a mono camera integrated to the robot. The proposed system is a 23-layer deep convolutional neural network (CNN) that is capable to estimate the pose of the robot in real time using a standard CPU. The dataset for the evaluation of the system was recorded inside a surgical human stomach model with realistic surface texture, softness, and surface liquid properties so that the pre-trained CNN architecture can be transferred confidently into a real endoscopic scenario. An average error of 7:1% and 3:4% for translation and rotation has been obtained, respectively. The results accomplished from the experiments demonstrate that a CNN pre-trained with raw 2D endoscopic images performs accurately inside the GI tract and is robust to various challenges posed by reflection distortions, lens imperfections, vignetting, noise, motion blur, low resolution, and lack of unique landmarks to track.

[1]  Jake J. Abbott,et al.  Managing the attractive magnetic force between an untethered magnetically actuated tool and a rotating permanent magnet , 2013, 2013 IEEE International Conference on Robotics and Automation.

[2]  Z. Liao,et al.  Indications and detection, completion, and retention rates of small-bowel capsule endoscopy: a systematic review. , 2010, Gastrointestinal endoscopy.

[3]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[4]  Stanislav Emelianov,et al.  Sonographic Elasticity Imaging of Acute and Chronic Deep Venous Thrombosis in Humans , 2006 .

[5]  M. Goenka,et al.  Capsule endoscopy: Present status and future expectation. , 2014, World journal of gastroenterology.

[6]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[7]  Jake J. Abbott,et al.  An omnidirectional electromagnet for remote manipulation , 2013, 2013 IEEE International Conference on Robotics and Automation.

[8]  C. Jia,et al.  Noninvasive ultrasound elasticity imaging (UEI) of Crohn's disease: animal model. , 2008, Ultrasound in medicine & biology.

[9]  G. Pan,et al.  Swallowable Wireless Capsule Endoscopy: Progress and Technical Challenges , 2011, Gastroenterology research and practice.

[10]  Tetsuya Nakamura,et al.  Capsule endoscopy: past, present, and future , 2008, Journal of Gastroenterology.

[11]  Weihua Li,et al.  A Review of Localization Systems for Robotic Endoscopic Capsules , 2012, IEEE Transactions on Biomedical Engineering.

[12]  Guang-Zhong Yang,et al.  Simultaneous Stereoscope Localization and Soft-Tissue Mapping for Minimal Invasive Surgery , 2006, MICCAI.

[13]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[14]  Metin Sitti,et al.  Biopsy using a Magnetic Capsule Endoscope Carrying, Releasing, and Retrieving Untethered Microgrippers , 2014, IEEE Transactions on Biomedical Engineering.

[15]  A. Juloski,et al.  Method for navigation and control of a magnetically guided capsule endoscope in the human stomach , 2012, 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[16]  Roberto Cipolla,et al.  PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[17]  Eric Diller,et al.  Biomedical Applications of Untethered Mobile Milli/Microrobots , 2015, Proceedings of the IEEE.

[18]  Weihua Li,et al.  A review of drug delivery systems for capsule endoscopy. , 2014, Advanced drug delivery reviews.

[19]  Yu Sun,et al.  Simultaneous Tracking, 3D Reconstruction and Deforming Point Detection for Stereoscope Guided Surgery , 2013, AE-CAI.

[20]  Alexandre Hostettler,et al.  ORBSLAM-Based Endoscope Tracking and 3D Reconstruction , 2016, CARE@MICCAI.

[21]  J. M. M. Montiel,et al.  EKF Monocular SLAM 3 D Modeling , Measuring and Augmented Reality from Endoscope Image Sequences , 2009 .

[22]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[23]  Metin Sitti,et al.  3-D Localization Method for a Magnetically Actuated Soft Capsule Endoscope and Its Applications , 2013, IEEE Transactions on Robotics.

[24]  Michael Talcott,et al.  Magnetically Controllable Gastrointestinal Steering of Video Capsules , 2011, IEEE Transactions on Biomedical Engineering.

[25]  M. Fluckiger,et al.  Ultrasound Emitter Localization in Heterogeneous Media , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[26]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).