Stereoscopic Near-Infrared Fluorescence Imaging: A Proof of Concept Toward Real-Time Depth Perception in Surgical Robotics

The increasing use of surgical robotics has provoked the necessity for new medical imaging methods. Many assistive surgical robotic systems influence the surgeon's movements based on a model of constraints and boundaries driven by anatomy. This study aims to demonstrate that Near-Infrared Fluorescence (NIRF) imaging could be applied in surgical applications to provide subsurface mapping of capillaries beneath soft tissue as a method for imaging active constraints. The manufacture of a system for imaging in the near-infrared wavelength range is presented, followed by a description of computational methods for stereo-post-processing and data acquisition and testing used to demonstrate that the proposed methods are viable. The results demonstrate that it is possible to use NIRF for the imaging of a capillary submersed up to 11 mm below a soft tissue phantom, over a range of angles from 0° through 45°. Phantom depth has been measured to an accuracy of ±3 mm and phantom angle to a constant accuracy of ±1.6°. These findings suggest that NIRF could be used for the next generation of medical imaging in surgical robotics and provide a basis for future research into real-time depth perception in the mapping of active constraints.

[1]  Khawaja H. Bilal,et al.  Indocyanine Green (ICG) in Urologic Surgery. , 2019, Urology.

[2]  J. Frangioni In vivo near-infrared fluorescence imaging. , 2003, Current opinion in chemical biology.

[3]  Stuart A. Bowyer,et al.  Dynamic frictional constraints for robot assisted surgery , 2013, 2013 World Haptics Conference (WHC).

[4]  Alwin Kienle,et al.  Broadband Optical Properties of Milk , 2017, Applied spectroscopy.

[5]  Su-Lin Lee,et al.  From medical images to minimally invasive intervention: Computer assistance for robotic surgery , 2010, Comput. Medical Imaging Graph..

[6]  Wieland B Huttner,et al.  A tunable refractive index matching medium for live imaging cells, tissues and model organisms , 2017, eLife.

[7]  Merlijn Hutteman,et al.  The clinical use of indocyanine green as a near‐infrared fluorescent contrast agent for image‐guided oncologic surgery , 2011, Journal of surgical oncology.

[8]  A. Vahrmeijer,et al.  2500 POSTER Identification and Image-guided Resection of Occult Superficial Liver Metastases Using Indocyanine Green and Near-infrared Fluorescence Imaging , 2011 .

[9]  Xiao Lu,et al.  Robust stereo matching with trinary cross color census and triple image-based refinements , 2017, EURASIP J. Adv. Signal Process..

[10]  Emanuele Trucco,et al.  Computer and Robot Vision , 1995 .

[11]  K. M. Buddhiraju,et al.  Modified Dual Winner Takes All Approach for Tri-Stereo Image Matching Using Disparity Space Images , 2017, Journal of the Indian Society of Remote Sensing.

[12]  Zbigniew Starosolski,et al.  Indocyanine green fluorescence in second near-infrared (NIR-II) window , 2017, PloS one.

[13]  Ferdinando Rodriguez y Baena,et al.  Dissipative Control for Physical Human–Robot Interaction , 2015, IEEE Transactions on Robotics.

[14]  Ka Wai Kwok Dynamic active constraints for robot assisted minimally invasive surgery , 2012 .

[15]  T van Doorn,et al.  Optical transmission properties of homogenised milk used as a phantom material in visible wavelength imaging. , 1995, Australasian physical & engineering sciences in medicine.

[16]  Cornelis J H van de Velde,et al.  Optimization of near-infrared fluorescent sentinel lymph node mapping for vulvar cancer. , 2012, American journal of obstetrics and gynecology.