CaDIS: Cataract Dataset for Image Segmentation

Video feedback provides a wealth of information about surgical procedures and is the main sensory cue for surgeons. Scene understanding is crucial to computer assisted interventions (CAI) and to post-operative analysis of the surgical procedure. A fundamental building block of such capabilities is the identification and localization of surgical instruments and anatomical structures through semantic segmentation. Deep learning has advanced semantic segmentation techniques in the recent years but is inherently reliant on the availability of labeled datasets for model training. This paper introduces a dataset for semantic segmentation of cataract surgery videos. The annotated images are part of the publicly available CATARACTS challenge dataset. In addition, we benchmark the performance of several state-of-the-art deep learning models for semantic segmentation on the presented dataset. The dataset is publicly available at this https URL .

[1]  Satoshi Kondo,et al.  CATARACTS: Challenge on automatic tool annotation for cataRACT surgery , 2019, Medical Image Anal..

[2]  Danail Stoyanov,et al.  EasyLabels: weak labels for scene segmentation in laparoscopic videos , 2019, International Journal of Computer Assisted Radiology and Surgery.

[3]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[4]  Danail Stoyanov,et al.  SurReal: enhancing Surgical simulation Realism using style transfer , 2018, BMVC.

[5]  Stefanie Speidel,et al.  Learning soft tissue behavior of organs for surgical navigation with convolutional neural networks , 2019, International Journal of Computer Assisted Radiology and Surgery.

[6]  Peter M. Full,et al.  Improving Surgical Training Phantoms by Hyperrealism: Deep Unpaired Image-to-Image Translation from Real Surgeries , 2018, MICCAI.

[7]  Andrew W. Senior,et al.  Long short-term memory recurrent neural network architectures for large scale acoustic modeling , 2014, INTERSPEECH.

[8]  Nicolai Schoch,et al.  Surgical Data Science: Enabling Next-Generation Surgery , 2017, ArXiv.

[9]  Orçun Göksel,et al.  Extending pretrained segmentation networks with additional anatomical structures , 2018, International Journal of Computer Assisted Radiology and Surgery.

[10]  Leo Joskowicz,et al.  Haptic computer-assisted patient-specific preoperative planning for orthopedic fractures surgery , 2015, International Journal of Computer Assisted Radiology and Surgery.

[11]  Masaru Ishii,et al.  Objective Assessment of Surgical Technical Skill and Competency in the Operating Room. , 2017, Annual review of biomedical engineering.

[12]  Russell H. Taylor,et al.  Data-Driven Visual Tracking in Retinal Microsurgery , 2012, MICCAI.

[13]  Nazneen Nazm,et al.  Posterior capsular rent: Prevention and management , 2017, Indian journal of ophthalmology.

[14]  Sotirios A. Tsaftaris,et al.  Medical Image Computing and Computer Assisted Intervention , 2017 .

[15]  Sébastien Ourselin,et al.  Real-Time Segmentation of Non-rigid Surgical Tools Based on Deep Learning and Tracking , 2016, CARE@MICCAI.

[16]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[17]  Håkon Olav Leira,et al.  Semantic segmentation and detection of mediastinal lymph nodes and anatomical structures in CT data for lung cancer staging , 2019, International Journal of Computer Assisted Radiology and Surgery.

[18]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  G. Marchal,et al.  Image segmentation: methods and applications in diagnostic radiology and nuclear medicine. , 1993, European journal of radiology.

[20]  Andru Putra Twinanda,et al.  EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos , 2016, IEEE Transactions on Medical Imaging.

[21]  Guang-Zhong Yang,et al.  A machine learning approach for real-time modelling of tissue deformation in image-guided neurosurgery , 2017, Artif. Intell. Medicine.

[22]  Lena Maier-Hein,et al.  2017 Robotic Instrument Segmentation Challenge , 2019, ArXiv.

[23]  Dong Liu,et al.  High-Resolution Representations for Labeling Pixels and Regions , 2019, ArXiv.

[24]  Judy E. Kim,et al.  Medical malpractice claims related to cataract surgery complicated by retained lens fragments (an American Ophthalmological Society thesis). , 2012, Transactions of the American Ophthalmological Society.

[25]  Danail Stoyanov,et al.  DeepPhase: Surgical Phase Recognition in CATARACTS Videos , 2018, MICCAI.

[26]  Danail Stoyanov,et al.  Surgical robotics beyond enhanced dexterity instrumentation: a survey of machine learning techniques and their role in intelligent and autonomous surgical actions , 2016, International Journal of Computer Assisted Radiology and Surgery.

[27]  C. Dolea,et al.  World Health Organization , 1949, International Organization.

[28]  George Papandreou,et al.  Rethinking Atrous Convolution for Semantic Image Segmentation , 2017, ArXiv.

[29]  Danail Stoyanov,et al.  Feature Aggregation Decoder for Segmenting Laparoscopic Scenes , 2019, OR/MLCN@MICCAI.

[30]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[32]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  W Hasler Pascal.  Essential Principles of Phacoemulsification , 2013 .

[36]  Zhanglin Wu,et al.  A combination of three-dimensional printing and computer-assisted virtual surgical procedure for preoperative planning of acetabular fracture reduction. , 2016, Injury.

[37]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[38]  R. Kikinis,et al.  Automated segmentation of MR images of brain tumors. , 2001, Radiology.

[39]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[40]  Yuning Jiang,et al.  Unified Perceptual Parsing for Scene Understanding , 2018, ECCV.

[41]  Sébastien Ourselin,et al.  Image Based Surgical Instrument Pose Estimation with Multi-class Labelling and Optical Flow , 2015, MICCAI.