Hierarchical Structure from Motion from Endoscopic Video

We present a hierarchical approach to structure from motion. This approach uses the notion of frame distance, which we define to be the median image displacement of tracked features. Image pairs with a high frame distance are used to initialise reconstruction, as they are expected to have significant associated camera motion and, therefore, give strong geometric constraints on the reconstruction. Additional frames are then added with progressively smaller frame distances to create denser reconstructions. We demonstrate this technique on endoscopic video, where there is a mix of rapid camera motion and periods where the camera is nearly stationary.

[1]  Darius Burschka,et al.  Scale-Invariant Registration of Monocular Endoscopic Images to CT-Scans for Sinus Surgery , 2004, MICCAI.

[2]  Charles Soussen,et al.  A novel 3D surface construction approach: Application to three-dimensional endoscopic data , 2010, 2010 IEEE International Conference on Image Processing.

[3]  Daniel Mirota,et al.  Robust motion estimation and structure recovery from endoscopic image sequences with an Adaptive Scale Kernel Consensus estimator , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Andrea Fusiello,et al.  Structure-and-motion pipeline on a hierarchical cluster tree , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[5]  A. Darzi,et al.  The effect of a second-generation 3D endoscope on the laparoscopic precision of novices and experienced surgeons , 1999, Surgical Endoscopy.

[6]  Christoph Schmalz,et al.  An endoscopic 3D scanner based on structured light , 2012, Medical Image Anal..

[7]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[9]  K. Deguchi,et al.  Shape reconstruction from an endoscope image by shape-from-shading technique for a point light source at the projection center , 1996, Proceedings of the Workshop on Mathematical Methods in Biomedical Image Analysis.

[10]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[11]  J. M. M. Montiel,et al.  Visual SLAM for Handheld Monocular Endoscope , 2014, IEEE Transactions on Medical Imaging.

[12]  Richard Szeliski,et al.  Skeletal graphs for efficient structure from motion , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Danail Stoyanov,et al.  A practical approach towards accurate dense 3D depth recovery for robotic laparoscopic surgery , 2005, Computer aided surgery : official journal of the International Society for Computer Aided Surgery.

[14]  W. Brent Seales,et al.  Endoscopic Video Texture Mapping on Pre-Built 3-D Anatomical Objects Without Camera Tracking , 2010, IEEE Transactions on Medical Imaging.

[15]  Branislav Jaramaz,et al.  A Multi-Image Shape-from-Shading Framework for Near-Lighting Perspective Endoscopes , 2009, International Journal of Computer Vision.

[16]  M. Figl,et al.  2D/3D registration of endoscopic ultrasound to CT volume data , 2008, Physics in medicine and biology.

[17]  William E. Higgins,et al.  Technique for registering 3D virtual CT images to endoscopic video , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[18]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[19]  Darius Burschka,et al.  Scale-invariant registration of monocular endoscopic images to CT-scans for sinus surgery , 2005, Medical Image Anal..

[20]  Richard Szeliski,et al.  Towards Internet-scale multi-view stereo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Elina Kari,et al.  Comparison of traditional 2‐dimensional endoscopic pituitary surgery with new 3‐dimensional endoscopic technology: intraoperative and early postoperative factors , 2012, International forum of allergy & rhinology.

[23]  Carlos Henrique Quartucci Forster,et al.  Towards 3D reconstruction of endoscope images using shape from shading , 2000, Proceedings 13th Brazilian Symposium on Computer Graphics and Image Processing (Cat. No.PR00878).

[24]  T. Schwartz,et al.  Three-dimensional Neurostereoendoscopy: Subjective and Objective Comparison to 2D , 2009, Minimally invasive neurosurgery : MIN.

[25]  Jianfei Liu,et al.  A stable optic-flow based method for tracking colonoscopy images , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[26]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[27]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[28]  Takahiro Ishikawa,et al.  The template update problem , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  S. Mills,et al.  Hierarchical structure from motion optical flow algorithms to harvest three-dimensional features from two-dimensional neuro-endoscopic images , 2015, Journal of Clinical Neuroscience.

[30]  A. Szold,et al.  Three-dimensional imaging improves surgical performance for both novice and experienced operators using the da Vinci Robot System. , 2007, American journal of surgery.

[31]  David Nistér,et al.  An efficient solution to the five-point relative pose problem , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..