EKF monocular SLAM with relocalization for laparoscopic sequences

In recent years, research on visual SLAM has produced robust algorithms providing, in real time at 30 Hz, both the 3D model of the observed rigid scene and the 3D camera motion using as only input the gathered image sequence. These algorithms have been extensively validated in rigid human-made environments -indoor and outdoor- showing robust performance in dealing with clutter, occlusions or sudden motions. Medical endoscopic sequences naturally pose a monocular SLAM problem: an unknown camera motion in an unknown environment. The corresponding map would be useful in providing 3D information to assist surgeons, to support augmented reality insertions or to be exploited by medical robots. In this paper we propose the combination EKF Monocular SLAM + 1-Point RANSAC + Randomised List Relocalization to process laparoscopic sequences -abdominal cavity images-. The sequences are challenging due to: 1) cluttering produced by tools; 2) sudden motions of the camera; 3) laparoscope frequently goes in and out of abdominal cavity; 4) tissue deformation caused by respiration, heartbeats and/or surgical tools. Real medical image sequences provide experimental validation.

[1]  Guang-Zhong Yang,et al.  Motion Compensated SLAM for Image Guided Surgery , 2010, MICCAI.

[2]  Ian D. Reid,et al.  Mapping Large Loops with a Single Hand-Held Camera , 2007, Robotics: Science and Systems.

[3]  J. M. M. Montiel,et al.  EKF Monocular SLAM 3 D Modeling , Measuring and Augmented Reality from Endoscope Image Sequences , 2009 .

[4]  Guang-Zhong Yang,et al.  Simultaneous Stereoscope Localization and Soft-Tissue Mapping for Minimal Invasive Surgery , 2006, MICCAI.

[5]  Javier Civera,et al.  1-Point RANSAC for extended Kalman filtering: Application to real-time structure from motion and visual odometry , 2010 .

[6]  Vincent Lepetit,et al.  Keypoint recognition using randomized trees , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Danail Stoyanov,et al.  A practical approach towards accurate dense 3D depth recovery for robotic laparoscopic surgery , 2005, Computer aided surgery : official journal of the International Society for Computer Aided Surgery.

[8]  Andrew J. Davison,et al.  Real-Time Spherical Mosaicing Using Whole Image Alignment , 2010, ECCV.

[9]  Chia-Hsiang Wu,et al.  Three-Dimensional Modeling From Endoscopic Video Using Geometric Constraints Via Feature Positioning , 2007, IEEE Transactions on Biomedical Engineering.

[10]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[11]  Yuan-Fang Wang,et al.  Toward automated model building from video in computer-assisted diagnoses in colonoscopy , 2007, SPIE Medical Imaging.

[12]  G. Klein,et al.  Parallel Tracking and Mapping for Small AR Workspaces , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[13]  Andrew J. Davison,et al.  Real-time simultaneous localisation and mapping with a single camera , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[14]  Olivier Stasse,et al.  MonoSLAM: Real-Time Single Camera SLAM , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Juan D. Tardós,et al.  Data association in stochastic mapping using the joint compatibility test , 2001, IEEE Trans. Robotics Autom..

[16]  Ian D. Reid,et al.  Real-Time SLAM Relocalisation , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[17]  Javier Civera,et al.  Unified Inverse Depth Parametrization for Monocular SLAM , 2006, Robotics: Science and Systems.

[18]  A. Davison,et al.  1-Point RANSAC for EKF Filtering . Application to Real-Time Structure from Motion and Visual Odometry , 2010 .

[19]  Javier Civera,et al.  1‐Point RANSAC for extended Kalman filtering: Application to real‐time structure from motion and visual odometry , 2010, J. Field Robotics.

[20]  Ève Coste-Manière,et al.  3D reconstruction of the operating field for image overlay in 3D-endoscopic surgery , 2001, Proceedings IEEE and ACM International Symposium on Augmented Reality.

[21]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Javier Civera,et al.  Inverse Depth Parametrization for Monocular SLAM , 2008, IEEE Transactions on Robotics.

[23]  Darius Burschka,et al.  Scale-Invariant Registration of Monocular Endoscopic Images to CT-Scans for Sinus Surgery , 2004, MICCAI.

[24]  David W. Murray,et al.  Improving the Agility of Keyframe-Based SLAM , 2008, ECCV.

[25]  Tom Drummond,et al.  Fusing points and lines for high performance tracking , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.