Applications of Image Processing in Laparoscopic Surgeries: An Overview

Laparoscopy is a minimally-invasive surgery using a few small incisions on the patient’s body to insert the tools and telescope and conduct the surgical operation. Laparoscopic video processing can be used to extract valuable knowledge and help the surgeons. We discuss the present and possible future role of processing laparoscopic videos. The various applications are categorized for image processing algorithms in laparoscopic surgeries including preprocessing video frames by laparoscopic image enhancement, telescope related applications (telescope position estimation, telescope motion estimation and compensation), surgical instrument related applications (surgical instrument detection and tracking), soft tissue related applications (soft tissue segmentation and deformation tracking) and high level applications such as safe actions in laparoscopic videos, summarization of laparoscopic videos, surgical task recognition and extracting knowledge using fusion techniques. Some different methods have been proposed previously for each of the mentioned applications using image processing. Applications of Image Processing in Laparoscopic Surgeries: An Overview

[1]  Yukio Kosugi,et al.  Neural Network Blood Vessel Detection in Laparoscopy , 2007 .

[2]  Sebastian Bodenstedt,et al.  Image-based tracking of the suturing needle during laparoscopic interventions , 2015, Medical Imaging.

[3]  Philippe Cinquin,et al.  Automatic Detection of Instruments in Laparoscopic Images: A First Step Towards High-level Command of Robotic Endoscopic Holders , 2007, The First IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, 2006. BioRob 2006..

[4]  Kai-Tai Song,et al.  Image tracking of laparoscopic instrument using spiking neural networks , 2013, 2013 13th International Conference on Control, Automation and Systems (ICCAS 2013).

[5]  D. Oleynikov,et al.  Real-time three-dimensional soft tissue reconstruction for laparoscopic surgery , 2012, Surgical Endoscopy.

[6]  K. Robert Lai,et al.  Improved global motion estimation via motion vector clustering for video stabilization , 2016, Eng. Appl. Artif. Intell..

[7]  Y. F. Wang,et al.  Automated instrument tracking in robotically assisted laparoscopic surgery. , 1995, Journal of image guided surgery.

[8]  Jung-Hwan Oh,et al.  Informative frame classification for endoscopy video , 2007, Medical Image Anal..

[9]  P. Curcillo,et al.  Single port access adrenalectomy. , 2008, Journal of endourology.

[10]  Zijian Zhao.  Real-time 3D visual tracking of laparoscopic instruments for robotized endoscope holder , 2014, Proceeding of the 11th World Congress on Intelligent Control and Automation.

[11]  Nassir Navab,et al.  Modeling and Segmentation of Surgical Workflow from Laparoscopic Video , 2010, MICCAI.

[12]  Klaus Schöffmann,et al.  Relevance Segmentation of Laparoscopic Videos , 2013, 2013 IEEE International Symposium on Multimedia.

[13]  Guang-Zhong Yang,et al.  Illumination position estimation for 3D soft-tissue reconstruction in robotic minimally invasive surgery , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[14]  A D'Alessio,et al.  One-Trocar Transumbilical Laparoscopic-Assisted Appendectomy in Children: Our Experience , 2002, European journal of pediatric surgery : official journal of Austrian Association of Pediatric Surgery ... [et al] = Zeitschrift fur Kinderchirurgie.

[15]  V. Agnus,et al.  Context-specific selection of algorithms for recursive feature tracking in endoscopic image using a new methodology , 2015, Comput. Medical Imaging Graph..

[16]  Su-Lin Lee,et al.  From medical images to minimally invasive intervention: Computer assistance for robotic surgery , 2010, Comput. Medical Imaging Graph..

[17]  Jung-Hwan Oh,et al.  Automatic real-time detection of endoscopic procedures using temporal features , 2012, Comput. Methods Programs Biomed..

[18]  G. Hirzinger,et al.  Real-time visual servoing for laparoscopic surgery. Controlling robot motion with color image segmentation , 1997, IEEE Engineering in Medicine and Biology Magazine.

[19]  Rutger M. Schols,et al.  Advanced intraoperative imaging methods for laparoscopic anatomy navigation: an overview , 2012, Surgical Endoscopy.

[20]  Alexandre Krupa,et al.  Combined image-based and depth visual servoing applied to robotized laparoscopic surgery , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[21]  Saowapak S. Thongvigitmanee,et al.  Adaptive Mean-Shift Kalman Tracking of Laparoscopic Instruments , 2012 .

[22]  Pablo Lamata,et al.  Laparoscopic video analysis for training and image-guided surgery , 2011, Minimally invasive therapy & allied technologies : MITAT : official journal of the Society for Minimally Invasive Therapy.

[23]  Tomoyuki Hiroyasu,et al.  Emphasizing mesenteric blood vessels in laparoscopic colon cancer surgery video images , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[24]  Guillaume-Alexandre Bilodeau,et al.  Segmentation of laparoscopic images: integrating graph-based segmentation and multistage region merging , 2005, The 2nd Canadian Conference on Computer and Robot Vision (CRV'05).

[25]  Beat P. Müller-Stich,et al.  Soft tissue navigation for laparoscopic partial nephrectomy , 2008, International Journal of Computer Assisted Radiology and Surgery.

[26]  J. Kaspersen,et al.  Laparoscopic navigation pointer for three-dimensional image–guided surgery , 2004, Surgical Endoscopy And Other Interventional Techniques.

[27]  B. Hannaford,et al.  Task decomposition of laparoscopic surgery for objective evaluation of surgical residents' learning curve using hidden Markov model. , 2002, Computer aided surgery : official journal of the International Society for Computer Aided Surgery.

[28]  Anupam Joshi,et al.  Detection of unsafe action from laparoscopic cholecystectomy video , 2012, IHI '12.

[29]  Adrien Bartoli,et al.  Deformable Shape-From-Motion in Laparoscopy using a Rigid Sliding Window , 2011, MIUA.

[30]  Ehud Rivlin,et al.  Online action recognition using covariance of shape and motion , 2014, Comput. Vis. Image Underst..

[31]  Makoto Hashizume,et al.  Objective Skill Evaluation for Laparoscopic Training Based on Motion Analysis , 2013, IEEE Transactions on Biomedical Engineering.

[32]  Gregory D. Hager,et al.  Surgical gesture classification from video and kinematic data , 2013, Medical Image Anal..

[33]  Kil Yeon Lee,et al.  Single port laparoscopic right hemicolectomy with D3 dissection for advanced colon cancer. , 2010, World journal of gastroenterology.

[34]  Toktam Khatibi,et al.  A Novel Unsupervised Approach for Minimally-invasive Video Segmentation , 2014, Journal of medical signals and sensors.

[35]  Zohreh Azimifar,et al.  Structured sparse representation for human action recognition , 2015, Neurocomputing.

[36]  Xiaoli Zhang,et al.  Application of visual tracking for robot-assisted laparoscopic surgery , 2002, J. Field Robotics.

[37]  Dimitrios K. Iakovidis,et al.  Reduction of capsule endoscopy reading times by unsupervised image mining , 2010, Comput. Medical Imaging Graph..

[38]  Toktam Khatibi,et al.  SIDF: A Novel Framework for Accurate Surgical Instrument Detection in Laparoscopic Video Frames , 2013 .

[39]  Jacques Marescaux,et al.  A computed stereoscopic method for laparoscopic surgery by using feature tracking , 2015, 2015 IEEE International Conference on Consumer Electronics - Taiwan.

[40]  Guang-Zhong Yang,et al.  Probabilistic Tracking of Affine-Invariant Anisotropic Regions , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Sandrine Voros,et al.  2D/3D Real-Time Tracking of Surgical Instruments Based on Endoscopic Image Processing , 2015, CARE@MICCAI.

[42]  Tatsuo Igarashi,et al.  Computer‐based endoscopic image‐processing technology for endourology and laparoscopic surgery , 2009, International journal of urology : official journal of the Japanese Urological Association.

[43]  Roberto A. Hexsel,et al.  Particle filtering in the Hough space for instrument tracking , 2012, Comput. Biol. Medicine.

[44]  Gwénolé Quellec,et al.  Real-time recognition of surgical tasks in eye surgery videos , 2014, Medical Image Anal..

[45]  M. Braga,et al.  Randomized clinical trial of laparoscopic versus open left colonic resection , 2010, The British journal of surgery.

[46]  Anupam Joshi,et al.  Video Summarization of Laparoscopic Cholecystectomies , 2009 .

[47]  Jihad H. Kaouk,et al.  Robotic Single-port Surgery: Paving the Way for the Future. , 2016, Urology.

[48]  Jenny Dankelman,et al.  Workflow mining and outlier detection from clinical activity logs , 2012, J. Biomed. Informatics.

[49]  Yung-Nien Sun,et al.  Automatic extraction and visualization of human inner structures from endoscopic image sequences , 2004, SPIE Medical Imaging.

[50]  Guang-Zhong Yang,et al.  Context specific descriptors for tracking deforming tissue , 2012, Medical Image Anal..

[51]  J. Marescaux,et al.  Augmented reality in laparoscopic surgical oncology. , 2011, Surgical oncology.

[52]  Philippe Cinquin,et al.  3D Tracking of Laparoscopic Instruments Using Statistical and Geometric Modeling , 2011, MICCAI.

[53]  Yuan-Fang Wang,et al.  Automated instrument tracking in robotically assisted laparoscopic surgery. , 1995 .

[54]  Shuai Wang,et al.  Scalable gastroscopic video summarization via similar-inhibition dictionary selection , 2016, Artif. Intell. Medicine.

[55]  Satoki Zenbutsu,et al.  Development of Blood Vessel Depth Displaying Method for Laparoscopic Surgery Guidance , 2013 .

[56]  Guang-Zhong Yang,et al.  Invisible Shadow for Navigation and Planning in Minimal Invasive Surgery , 2005, MICCAI.

[57]  Pierre Graebling,et al.  Real-time segmentation of surgical instruments inside the abdominal cavity using a joint hue saturation color feature , 2005, Real Time Imaging.

[58]  J. A. Sánchez-Margallo,et al.  EVA: Laparoscopic Instrument Tracking Based on Endoscopic Video Analysis for Psychomotor Skills Assessment , 2013, Surgical Endoscopy.

[59]  Guang-Zhong Yang,et al.  Soft tissue deformation tracking for robotic assisted minimally invasive surgery , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[60]  Alireza Mirbagheri,et al.  Automatic tracking of laparoscopic instruments for autonomous control of a cameraman robot , 2016, Minimally invasive therapy & allied technologies : MITAT : official journal of the Society for Minimally Invasive Therapy.

[61]  Xiangtao Zheng,et al.  A discriminative representation for human action recognition , 2016, Pattern Recognit..

[62]  Rolf-Rainer Grigat,et al.  Closed Contour Specular Reflection Segmentation in Laparoscopic Images , 2013, Int. J. Biomed. Imaging.

[63]  G Navarra,et al.  One‐wound laparoscopic cholecystectomy , 1997, The British journal of surgery.

[64]  Rajesh Aggarwal,et al.  Laparoscopic task recognition using Hidden Markov Models. , 2005, Studies in health technology and informatics.

[65]  Patricia Sánchez González,et al.  Laparoscopic image analysis for automatic tracking of surgical tools , 2008 .

[66]  Nassir Navab,et al.  A Boosted Segmentation Method for Surgical Workflow Analysis , 2007, MICCAI.

[67]  Jianfang Dou,et al.  Robust human action recognition based on spatio-temporal descriptors and motion temporal templates , 2014 .

[68]  T. Jarrett,et al.  30-Day Morbidity and Mortality Outcomes of Prolonged Minimally Invasive Kidney Procedures Compared with Shorter Open Procedures: National Surgical Quality Improvement Program Analysis. , 2015, Journal of endourology.

[69]  Gregory D. Hager,et al.  Towards automatic skill evaluation: detection and segmentation of robot-assisted surgical motions. , 2006 .

[70]  Andrew S. Wu,et al.  Single Port Access (SPA™) laparoscopic ventral hernia repair: initial report of 30 cases , 2009, Surgical Endoscopy.

[71]  Max Q.-H. Meng,et al.  Computer-based detection of bleeding and ulcer in wireless capsule endoscopy images by chromaticity moments , 2009, Comput. Biol. Medicine.

[72]  Byeong-Seok Shin,et al.  Reducing redundancy in wireless capsule endoscopy videos , 2013, Comput. Biol. Medicine.

[73]  Michael C. Yip,et al.  Real-Time Methods for Long-Term Tissue Feature Tracking in Endoscopic Scenes , 2012, IPCAI.

[74]  Yoshito Mekada,et al.  Automatic detection of informative frames from wireless capsule endoscopy images , 2010, Medical Image Anal..

[75]  G. Ballantyne.  Robotic surgery, telerobotic surgery, telepresence, and telementoring , 2002, Surgical Endoscopy And Other Interventional Techniques.

[76]  David J. Hawkes,et al.  Progressive internal landmark registration for surgical navigation in laparoscopic gastrectomy for gastric cancer , 2016, International Journal of Computer Assisted Radiology and Surgery.