Real-Time Dense Stereo Reconstruction Using Convex Optimisation with a Cost-Volume for Image-Guided Robotic Surgery

Reconstructing the depth of stereo-endoscopic scenes is an important step in providing accurate guidance in robotic-assisted minimally invasive surgery. Stereo reconstruction has been studied for decades but remains a challenge in endoscopic imaging. Current approaches can easily fail to reconstruct an accurate and smooth 3D model due to textureless tissue appearance in the real surgical scene and occlusion by instruments. To tackle these problems, we propose a dense stereo reconstruction algorithm using convex optimisation with a cost-volume to efficiently and effectively reconstruct a smooth model while maintaining depth discontinuity. The proposed approach has been validated by quantitative evaluation using simulation and real phantom data with known ground truth. We also report qualitative results from real surgical images. The algorithm outperforms state of the art methods and can be easily parallelised to run in real-time on recent graphics hardware.

[1]  Sebastian Bodenstedt,et al.  Dense GPU-enhanced surface reconstruction from stereo endoscopic images for intraoperative registration. , 2012, Medical physics.

[2]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[3]  Gregory Hager,et al.  Vision-based navigation in image-guided interventions. , 2011, Annual review of biomedical engineering.

[4]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[5]  Nassir Navab,et al.  Medical Image Computing and Computer-Assisted Intervention - MICCAI 2010, 13th International Conference, Beijing, China, September 20-24, 2010, Proceedings, Part III , 2010, MICCAI.

[6]  Hervé Delingette,et al.  Medical Image Computing and Computer-Assisted Intervention – MICCAI 2012 , 2012, Lecture Notes in Computer Science.

[7]  Guang-Zhong Yang,et al.  Real-Time Stereo Reconstruction in Robotically Assisted Minimally Invasive Surgery , 2010, MICCAI.

[8]  Pierre Hellier,et al.  Level Set Methods in an EM Framework for Shape Classification and Estimation , 2004, International Conference on Medical Image Computing and Computer-Assisted Intervention.

[9]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[10]  Daniel Cremers,et al.  Large displacement optical flow computation withoutwarping , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[11]  Danail Stoyanov,et al.  Stereoscopic Scene Flow for Robotic Assisted Minimally Invasive Surgery , 2012, MICCAI.

[12]  Philippe Poignet,et al.  Towards robust 3D visual tracking for motion compensation in beating heart surgery , 2011, Medical Image Anal..

[13]  Guang-Zhong Yang,et al.  Three-Dimensional Tissue Deformation Recovery and Tracking , 2010, IEEE Signal Processing Magazine.

[14]  Cristian A. Linte,et al.  Augmented Environments for Computer-Assisted Interventions , 2012, Lecture Notes in Computer Science.

[15]  Carsten Rother,et al.  Fast cost-volume filtering for visual correspondence and beyond , 2011, CVPR 2011.

[16]  Gregory D. Hager,et al.  Stereo-Based Endoscopic Tracking of Cardiac Surface Deformation , 2004, MICCAI.

[17]  Dongbin Chen,et al.  2D/3D Registration of a Preoperative Model with Endoscopic Video Using Colour-Consistency , 2011, AE-CAI.

[18]  Daniel Cremers,et al.  Anisotropic Huber-L1 Optical Flow , 2009, BMVC.

[19]  Ève Coste-Manière,et al.  Towards endoscopic augmented reality for robotically assisted minimally invasive cardiac surgery , 2001, Proceedings International Workshop on Medical Imaging and Augmented Reality.