Floating autostereoscopic 3D display with multidimensional images for telesurgical visualization

PurposeWe propose a combined floating autostereoscopic three-dimensional (3D) display approach for telesurgical visualization, which could reproduce live surgical scene in a realistic and intuitive manner.MethodsA polyhedron-shaped 3D display device is developed for spatially floating autostereoscopic 3D image. Integral videography (IV) technique is adopted to generate real-time 3D images. Combined two-dimensional (2D) and 3D displays are presented floatingly around the center of the display device through reflection of semitransparent mirrors. Intra-operative surgery information is fused and updated in the 3D display, so that telesurgical visualization could be enhanced remotely.ResultsThe experimental results showed that our approach can achieve a combined floating autostereoscopic display that presents 2D and 3D fusion images. The glasses-free IV 3D display has full parallax and can be observed by multiple persons from surrounding areas at the same time. Furthermore, real-time surgical scene could be presented and updated in a realistic and intuitive visualization platform. It is shown that the proposed method is feasible for facilitating telesurgical visualization.ConclusionThe proposed floating autostereoscopic display device presents surgical information in an efficient form, so as to enhance operative cooperation and efficiency during operation. Combined presentation of imaging information is promising for medical applications.

[1]  Nobuhiko Hata,et al.  Scalable high-resolution integral videography autostereoscopic display with a seamless multiprojection system. , 2005, Applied optics.

[2]  Stavros A. Antoniou,et al.  A comprehensive review of telementoring applications in laparoscopic general surgery , 2012, Surgical Endoscopy.

[3]  K M Irion,et al.  Systems workplace for endoscopic surgery , 2000, Minimally invasive therapy & allied technologies : MITAT : official journal of the Society for Minimally Invasive Therapy.

[4]  Hongen Liao,et al.  Fast image mapping of endoscopic image mosaics with three-dimensional ultrasound image for intrauterine fetal surgery , 2009, Minimally invasive therapy & allied technologies : MITAT : official journal of the Society for Minimally Invasive Therapy.

[5]  M. Curet,et al.  Single-port robotic cholecystectomy: results from a first human use clinical study of the new da Vinci single-site surgical platform. , 2011, Archives of surgery.

[6]  Yuncheng You,et al.  Video‐based 3D reconstruction, laparoscope localization and deformation recovery for abdominal minimally invasive surgery: a survey , 2016, The international journal of medical robotics + computer assisted surgery : MRCAS.

[7]  Masahiro Suzuki,et al.  Usefulness of a glass-free medical three-dimensional autostereoscopic display in neurosurgery , 2014, International Journal of Computer Assisted Radiology and Surgery.

[8]  Jason G Newman,et al.  Robotics and telesurgery in otolaryngology. , 2011, Otolaryngologic clinics of North America.

[9]  Baiquan Su,et al.  360 degree viewable floating autostereoscopic display using integral photography and multiple semitransparent mirrors. , 2015, Optics express.

[10]  Nobuhiko Hata,et al.  Surgical navigation by autostereoscopic image overlay of integral videography , 2004, IEEE Transactions on Information Technology in Biomedicine.

[11]  Ichiro Sakuma,et al.  Augmented Reality Navigation With Automatic Marker-Free Image Registration Using 3-D Image Overlay for Dental Surgery , 2014, IEEE Transactions on Biomedical Engineering.

[12]  Sallie J. Weaver,et al.  Making health care safer II: an updated critical analysis of the evidence for patient safety practices. , 2013, Evidence report/technology assessment.

[13]  Nobuhiko Hata,et al.  High-quality integral videography using a multiprojector. , 2004, Optics express.

[14]  Jacques Marescaux,et al.  Transatlantic robot-assisted telesurgery , 2001, Nature.

[15]  C Zindel.  System solutions for the integration of devices in the OR , 2000, Minimally invasive therapy & allied technologies : MITAT : official journal of the Society for Minimally Invasive Therapy.

[16]  H. Miyake,et al.  Novel telementoring system for robot-assisted radical prostatectomy: impact on the learning curve. , 2014, Urology.

[17]  Richard D Bucholz,et al.  Operating room integration and telehealth. , 2011, Acta neurochirurgica. Supplement.

[18]  Blake Hannaford,et al.  The RAVEN: Design and Validation of a Telesurgery System , 2009, Int. J. Robotics Res..

[19]  John Kenneth Salisbury,et al.  The Intuitive/sup TM/ telesurgery system: overview and application , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[20]  Byung-Gook Lee,et al.  Resolution-enhanced integral imaging in focal mode with a time-multiplexed electrical mask array. , 2014, Optics express.

[21]  Andreas Abildgaard,et al.  An autostereoscopic 3D display can improve visualization of 3D models from intracranial MR angiography , 2010, International Journal of Computer Assisted Radiology and Surgery.