Stable Image Registration for In-Vivo Fetoscopic Panorama Reconstruction

A Twin-to-Twin Transfusion Syndrome (TTTS) is a condition that occurs in about 10% of pregnancies involving monochorionic twins. This complication can be treated with fetoscopic laser coagulation. The procedure could greatly benefit from panorama reconstruction to gain an overview of the placenta. In previous work we investigated which steps could improve the reconstruction performance for an in-vivo setting. In this work we improved this registration by proposing a stable region detection method as well as extracting matchable features based on a deep-learning approach. Finally, we extracted a measure for the image registration quality and the visibility condition. With experiments we show that the image registration performance is increased and more constant. Using these methods a system can be developed that supports the surgeon during the surgery, by giving feedback and providing a more complete overview of the placenta.

[1]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Steven M. Seitz,et al.  Rectified Surface Mosaics , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[3]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[4]  Gregory D. Hager,et al.  Real-Time Endoscopic Mosaicking , 2006, MICCAI.

[5]  Vincent Lepetit,et al.  LIFT: Learned Invariant Feature Transform , 2016, ECCV.

[6]  Hongen Liao,et al.  Fast Image Mapping of Endoscopic Image Mosaics with Three-Dimensional Ultrasound Image for Intrauterine Treatment of Twin-to-Twin Transfusion Syndrome , 2008, MIAR.

[7]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  P. Rousseeuw.  Least Median of Squares Regression , 1984 .

[9]  Eric J. Seibel,et al.  Surface Mosaics of the Bladder Reconstructed From Endoscopic Video for Automated Surveillance , 2012, IEEE Transactions on Biomedical Engineering.

[10]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[11]  Philippe C. Cattin,et al.  Mosaicing of Endoscopic Placenta Images , 2006, GI Jahrestagung.

[12]  Toshio Chiba,et al.  Image Registration for Placenta Reconstruction , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[13]  Iasonas Kokkinos,et al.  Discriminative Learning of Deep Convolutional Feature Point Descriptors , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[14]  Vincent Lepetit,et al.  TILDE: A Temporally Invariant Learned DEtector , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Sébastien Ourselin,et al.  A Combined EM and Visual Tracking Probabilistic Model for Robust Mosaicking: Application to Fetoscopy , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[16]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[17]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[18]  Jan Deprest,et al.  The vascular anastomoses in monochorionic twin pregnancies and their clinical consequences. , 2012, American journal of obstetrics and gynecology.

[19]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..