Surgical Data Science: A Consensus Perspective

Surgical data science is a scientific discipline with the objective of improving the quality of interventional healthcare and its value through capturing, organization, analysis, and modeling of data. The goal of the 1st workshop on Surgical Data Science was to bring together researchers working on diverse topics in surgical data science in order to discuss existing challenges, potential standards and new research directions in the field. Inspired by current open space and think tank formats, it was organized in June 2016 in Heidelberg. While the first day of the workshop, which was dominated by interactive sessions, was open to the public, the second day was reserved for a board meeting on which the information gathered on the public day was processed by (1) discussing remaining open issues, (2) deriving a joint definition for surgical data science and (3) proposing potential strategies for advancing the field. This document summarizes the key findings.

[1]  Klaus H. Maier-Hein,et al.  Exploiting the potential of unlabeled endoscopic video data with self-supervised learning , 2017, International Journal of Computer Assisted Radiology and Surgery.

[2]  Sébastien Ourselin,et al.  The NifTK software platform for image-guided interventions: platform overview and NiftyLink messaging , 2014, International Journal of Computer Assisted Radiology and Surgery.

[3]  Joachim Denzler,et al.  Automatic Classification of Cancerous Tissue in Laserendomicroscopy Images of the Oral Cavity using Deep Learning , 2017, Scientific Reports.

[4]  Lei Xing,et al.  Development of an autonomous treatment planning strategy for radiation therapy with effective use of population‐based prior data , 2017, Medical physics.

[5]  Jenny Dankelman,et al.  A RFID Specific Participatory Design Approach to Support Design and Implementation of Real-Time Location Systems in the Operating Room , 2014, Journal of Medical Systems.

[6]  Nassir Navab,et al.  Towards Systematic Usability Evaluations for the OR: An Introduction to OR-Use Framework , 2012, IPCAI.

[7]  Rüdiger Dillmann,et al.  Bridging the gap between formal and experience-based knowledge for context-aware laparoscopy , 2016, International Journal of Computer Assisted Radiology and Surgery.

[8]  Octavian Prostean,et al.  Brain-Writing Vs. Brainstorming Case Study For Power Engineering Education , 2015 .

[9]  Yangyong Zhu,et al.  Defining Data Science , 2015, ArXiv.

[10]  Henry C. Lin,et al.  JHU-ISI Gesture and Skill Assessment Working Set ( JIGSAWS ) : A Surgical Activity Dataset for Human Motion Modeling , 2014 .

[11]  Gregory D. Hager,et al.  Recognizing Surgical Activities with Recurrent Neural Networks , 2016, MICCAI.

[12]  Patrick Philipp.  Sequential Decision Making with Medical Interpretation Algorithms in the Semantic Web , 2015, ESWC.

[13]  Robert Hauser,et al.  CancerLinQ and the future of cancer care. , 2013, American Society of Clinical Oncology educational book. American Society of Clinical Oncology. Annual Meeting.

[14]  Milan Sonka,et al.  3D Slicer as an image computing platform for the Quantitative Imaging Network. , 2012, Magnetic resonance imaging.

[15]  Germain Forestier,et al.  Unsupervised Trajectory Segmentation for Surgical Gesture Recognition in Robotic Training , 2016, IEEE Transactions on Biomedical Engineering.

[16]  Charles E. Kahn,et al.  Biomedical imaging ontologies: A survey and proposal for future work , 2015, Journal of pathology informatics.

[17]  J. Hall,et al.  See One, Do One, Teach One , 1999, Pediatrics.

[18]  José Luis Rojo-Álvarez,et al.  Data-driven Temporal Prediction of Surgical Site Infection , 2015, AMIA.

[19]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[20]  Nicolai Schoch,et al.  The Medical Simulation Markup Language - Simplifying the Biomechanical Modeling Workflow , 2014, MMVR.

[21]  Sean Bechhofer,et al.  OWL: Web Ontology Language , 2009, Encyclopedia of Database Systems.

[22]  Carla M. Pugh,et al.  Advanced Engineering Technology for Measuring Performance. , 2015, The Surgical clinics of North America.

[23]  Guang-Zhong Yang,et al.  Vision-based deformation recovery for intraoperative force estimation of tool–tissue interaction for neurosurgery , 2016, International Journal of Computer Assisted Radiology and Surgery.

[24]  D. Boyd,et al.  CRITICAL QUESTIONS FOR BIG DATA , 2012 .

[25]  Simon Weidert,et al.  Development and Procedural Evaluation of Immersive Medical Simulation Environments , 2013, IPCAI.

[26]  Rüdiger Dillmann,et al.  LapOntoSPM: an ontology for laparoscopic surgeries and its application to surgical phase recognition , 2015, International Journal of Computer Assisted Radiology and Surgery.

[27]  Gregory D. Hager,et al.  Surgical Phase Recognition: from Instrumented ORs to Hospitals Around the World , 2016 .

[28]  Ying Chen,et al.  IBM Watson: How Cognitive Computing Can Be Applied to Big Data Challenges in Life Sciences Research. , 2016, Clinical therapeutics.

[29]  M. Ashburner,et al.  Gene Ontology: tool for the unification of biology , 2000, Nature Genetics.

[30]  Ramin Shahidi,et al.  Validation of medical image processing in image-guided therapy , 2002, IEEE Transactions on Medical Imaging.

[31]  D. Louis Collins,et al.  Multi-site study of surgical practice in neurosurgery based on surgical process models , 2013, J. Biomed. Informatics.

[32]  Narges Ahmidi,et al.  Task-Level vs. Segment-Level Quantitative Metrics for Surgical Skill Assessment. , 2016, Journal of surgical education.

[33]  Guang-Zhong Yang,et al.  Autonomous Ultrasound-Guided Tissue Dissection , 2015, MICCAI.

[34]  Andru Putra Twinanda,et al.  Data-driven spatio-temporal RGBD feature encoding for action recognition in operating rooms , 2015, International Journal of Computer Assisted Radiology and Surgery.

[35]  Nikos Paragios,et al.  Deformable Medical Image Registration: A Survey , 2013, IEEE Transactions on Medical Imaging.

[36]  Keno März,et al.  Toward knowledge-based liver surgery: holistic information processing for surgical decision support , 2015, International Journal of Computer Assisted Radiology and Surgery.

[37]  Andru Putra Twinanda,et al.  EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos , 2016, IEEE Transactions on Medical Imaging.

[38]  V. Agnus,et al.  A step towards stereotactic navigation during pelvic surgery: 3D nerve topography , 2018, Surgical Endoscopy.

[39]  Yixuan Yuan,et al.  Deep learning for polyp recognition in wireless capsule endoscopy images , 2017, Medical physics.

[40]  Patrick C. Sanger,et al.  Diagnosing Surgical Site Infection Using Wound Photography: A Scenario-Based Study. , 2017, Journal of the American College of Surgeons.

[41]  Lena Maier-Hein,et al.  Can Masses of Non-Experts Train Highly Accurate Image Classifiers? - A Crowdsourcing Approach to Instrument Segmentation in Laparoscopic Images , 2014, MICCAI.

[42]  Danail Stoyanov,et al.  Vision‐based and marker‐less surgical tool detection and tracking: a review of the literature , 2017, Medical Image Anal..

[43]  Paulo Jorge Sequeira Gonçalves,et al.  Ontologies Applied to Surgical Robotics , 2015, ROBOT.

[44]  Nassir Navab,et al.  Personalized, relevance-based Multimodal Robotic Imaging and augmented reality for Computer Assisted Interventions , 2016, Medical Image Anal..

[45]  J. Birkmeyer,et al.  Surgical skill and complication rates after bariatric surgery. , 2013, The New England journal of medicine.

[46]  Kensaku Mori,et al.  Advanced Endoscopic Navigation: Surgical Big Data, Methodology, and Applications. , 2018, Annual review of biomedical engineering.

[47]  P. Jannin,et al.  Assessment of Image-Guided Interventions , 2008 .

[48]  Paul Aljabar,et al.  Clinical evaluation of atlas and deep learning based automatic contouring for lung cancer. , 2017, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[49]  C. Langlotz.  RadLex: a new method for indexing online educational materials. , 2006, Radiographics : a review publication of the Radiological Society of North America, Inc.

[50]  Raghunath Nambiar,et al.  A look at challenges and opportunities of Big Data analytics in healthcare , 2013, 2013 IEEE International Conference on Big Data.

[51]  M. Giger,et al.  Breast image analysis for risk assessment, detection, diagnosis, and treatment of cancer. , 2013, Annual review of biomedical engineering.

[52]  Septimiu E. Salcudean,et al.  On the feasibility of heart motion compensation on the daVinci® surgical robot for coronary artery bypass surgery: Implementation and user studies , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[53]  Thomas Schlich,et al.  Patient choice and the history of minimally invasive surgery , 2016, The Lancet.

[54]  Guang-Zhong Yang,et al.  Spatio-spectral classification of hyperspectral images for brain cancer detection during surgical operations , 2018, PloS one.

[55]  Ana Silva,et al.  Why facilitate patient access to medical records. , 2007, Studies in health technology and informatics.

[56]  A. Zwitter.  Big Data ethics , 2014, Big Data Soc..

[57]  Shamik Mascharak,et al.  Detecting oropharyngeal carcinoma using multispectral, narrow‐band imaging and machine learning , 2018, The Laryngoscope.

[58]  Johan Montagnat,et al.  NeuroLOG: sharing neuroimaging data using an ontology-based federated approach. , 2011, AMIA ... Annual Symposium proceedings. AMIA Symposium.

[59]  M. Ashburner,et al.  The OBO Foundry: coordinated evolution of ontologies to support biomedical data integration , 2007, Nature Biotechnology.

[60]  Russell H. Taylor,et al.  System for robot-assisted real-time laparoscopic ultrasound elastography , 2012, Medical Imaging.

[61]  William B. Lober,et al.  Prognostics of Surgical Site Infections using Dynamic Health Data , 2016, J. Biomed. Informatics.

[62]  Jianqiang Li,et al.  A hybrid solution for privacy preserving medical data sharing in the cloud environment , 2015, Future Gener. Comput. Syst..

[63]  Russell H. Taylor,et al.  Surgical data science for next-generation interventions , 2017, Nature Biomedical Engineering.

[64]  Klaus H. Maier-Hein,et al.  Physiological Parameter Estimation from Multispectral Images Unleashed , 2017, MICCAI.

[65]  Masaru Ishii,et al.  Objective Assessment of Surgical Technical Skill and Competency in the Operating Room. , 2017, Annual review of biomedical engineering.

[66]  Gero Strauß,et al.  Workflow analysis for mechatronic and imaging assistance in head surgery , 2004, CARS.

[67]  Guang-Zhong Yang,et al.  Real-Time 3D Tracking of Articulated Tools for Robotic Surgery , 2016, MICCAI.

[68]  Gregory D. Hager,et al.  A study of crowdsourced segment-level surgical skill assessment using pairwise rankings , 2015, International Journal of Computer Assisted Radiology and Surgery.

[69]  Pierre Jannin,et al.  Model for defining and reporting reference-based validation protocols in medical image processing , 2006, International Journal of Computer Assisted Radiology and Surgery.

[70]  Nicolai Schoch,et al.  Towards an open-source semantic data infrastructure for integrating clinical and scientific data in cognition-guided surgery , 2016, SPIE Medical Imaging.

[71]  Tharindu De Silva,et al.  Automatic Localization of Target Vertebrae in Spine Surgery: Clinical Evaluation of the LevelCheck Registration Algorithm , 2015, Spine.

[72]  Andrew Lippman,et al.  MedRec: Using Blockchain for Medical Data Access and Permission Management , 2016, 2016 2nd International Conference on Open and Big Data (OBD).

[73]  Masaru Ishii,et al.  Robotic Path Planning for Surgeon Skill Evaluation in Minimally-Invasive Sinus Surgery , 2012, MICCAI.

[74]  Nicolas Padoy,et al.  Articulated clinician detection using 3D pictorial structures on RGB‐D data , 2016, Medical Image Anal..

[75]  Anand Malpani.  Automated Virtual Coach for Surgical Training , 2017 .

[76]  Thomas Neumuth,et al.  Analysis of surgical intervention populations using generic surgical process models , 2010, International Journal of Computer Assisted Radiology and Surgery.

[77]  Paolo Fiorini,et al.  EuRoSurge Workflow: From ontology to surgical task execution , 2013 .

[78]  Gilmer Valdes,et al.  Clinical decision support of radiotherapy treatment planning: A data-driven machine learning strategy for patient-specific dosimetric decision making. , 2017, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[79]  João Paulo Costa,et al.  Semantic web and decision support systems , 2016, J. Decis. Syst..

[80]  Marco Nolden,et al.  The Medical Imaging Interaction Toolkit , 2005, Medical Image Anal..

[81]  Alex A. Kane,et al.  Comparison of an unsupervised machine learning algorithm and surgeon diagnosis in the clinical differentiation of metopic craniosynostosis and benign metopic ridge , 2018, Scientific Reports.

[82]  Nassir Navab,et al.  Statistical modeling and recognition of surgical workflow , 2012, Medical Image Anal..

[83]  M. Mack,et al.  Minimally invasive and robotic surgery. , 2001, JAMA.

[84]  Guoqiang Han,et al.  Quantitative analysis of patients with celiac disease by video capsule endoscopy: A deep learning method , 2017, Comput. Biol. Medicine.

[85]  B. Nordlinger,et al.  Where does pelvic nerve injury occur during rectal surgery for cancer? , 2011, Colorectal disease : the official journal of the Association of Coloproctology of Great Britain and Ireland.

[86]  Sergio Gomez Colmenarejo,et al.  Hybrid computing using a neural network with dynamic external memory , 2016, Nature.

[87]  Lena Maier-Hein,et al.  First clinical use of the EchoTrack guidance approach for radiofrequency ablation of thyroid gland nodules , 2017, International Journal of Computer Assisted Radiology and Surgery.

[88]  Gregory D. Hager,et al.  A Dataset and Benchmarks for Segmentation and Recognition of Gestures in Robotic Surgery , 2017, IEEE Transactions on Biomedical Engineering.

[89]  Tomoharu Kiyuna,et al.  Automated histological classification of whole-slide images of gastric biopsy specimens , 2018, Gastric Cancer.

[90]  Pierre Jannin,et al.  Surgical process modelling: a review , 2014, International Journal of Computer Assisted Radiology and Surgery.

[91]  Heinz U Lemke,et al.  Computer-assisted radiology. , 1988, Academic radiology.

[92]  Blake Hannaford,et al.  Semi-autonomous simulated brain tumor ablation with RAVENII Surgical Robot using behavior tree , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[93]  Nassir Navab,et al.  Recovery of Surgical Workflow Without Explicit Models , 2006, MICCAI.

[94]  Thomas E. Nichols,et al.  Best practices in data analysis and sharing in neuroimaging using MRI , 2017, Nature Neuroscience.

[95]  Kevin Cleary,et al.  OR 2020 workshop report: Operating room of the future , 2005 .

[96]  Barry Smith,et al.  SNAP and SPAN: Towards Dynamic Spatial Ontology , 2004, Spatial Cogn. Comput..

[97]  Nassir Navab,et al.  AggNet: Deep Learning From Crowds for Mitosis Detection in Breast Cancer Histology Images , 2016, IEEE Trans. Medical Imaging.

[98]  Nicolai Schoch,et al.  Comprehensive patient-specific information preprocessing for cardiac surgery simulations , 2016, International Journal of Computer Assisted Radiology and Surgery.

[99]  Andrea De Mauro,et al.  What is big data? A consensual definition and a review of key research topics , 2015, AIP Conference Proceedings.

[100]  Lingling Zhang,et al.  Some Comments on Big Data and Data Science , 2014 .

[101]  Markus W. Büchler,et al.  Surgical evaluation and knowledge transfer—methods of clinical research in surgery , 2012, Langenbeck's Archives of Surgery.

[102]  Heinz Wörn,et al.  Workflow analysis and surgical phase recognition in minimally invasive surgery , 2012, 2012 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[103]  Bernard R Bendok,et al.  Machine Learning as a Potential Solution for Shift During Stereotactic Brain Surgery. , 2018, Neurosurgery.

[104]  Towards an Intelligent Framework for Personalized Simulation-enhanced Surgery Assistance: Linking a Simulation Ontology to a Reinforcement Learning Algorithm for Calibration of Numerical Simulations , 2017 .

[105]  Patrick C. Sanger,et al.  A Prognostic Model of Surgical Site Infection Using Daily Clinical Wound Assessment. , 2016, Journal of the American College of Surgeons.

[106]  J R Siewert,et al.  Preoperative assessment of lymph node metastases in patients with gastric cancer: evaluation of the Maruyama computer program , 1992, The British journal of surgery.

[107]  Simon Weidert,et al.  Vertebroplasty Performance on Simulator for 19 Surgeons Using Hierarchical Task Analysis , 2015, IEEE Transactions on Medical Imaging.

[108]  Martin Kasparick,et al.  OR.NET: multi-perspective qualitative evaluation of an integrated operating room based on IEEE 11073 SDC , 2017, International Journal of Computer Assisted Radiology and Surgery.

[109]  M. Sasako,et al.  Lymph node metastases of gastric cancer. General pattern in 1931 patients. , 1989, Annals of surgery.

[110]  Teodor P. Grantcharov,et al.  Using Data to Enhance Performance and Improve Quality and Safety in Surgery. , 2017, JAMA surgery.

[111]  Lena Maier-Hein,et al.  Robust near real-time estimation of physiological parameters from megapixel multispectral images with inverse Monte Carlo and random forest regression , 2016, International Journal of Computer Assisted Radiology and Surgery.

[112]  João Manuel R S Tavares,et al.  Medical image registration: a review , 2014, Computer methods in biomechanics and biomedical engineering.

[113]  Jun Okamoto,et al.  SCOT (Smart Cyber Operating Theater) project: Advanced medical information analyzer for guidance of the surgical procedures , 2012 .

[114]  Julian M Goldman,et al.  Introducing new technology into the operating room: measuring the impact on job performance and satisfaction. , 2005, Surgery.

[115]  Alex Rubinsteyn,et al.  Using a Machine Learning Approach to Predict Outcomes after Radiosurgery for Cerebral Arteriovenous Malformations , 2016, Scientific Reports.

[116]  Nassir Navab,et al.  Robust colonoscope tracking method for colon deformations utilizing coarse-to-fine correspondence findings , 2016, International Journal of Computer Assisted Radiology and Surgery.

[117]  Andrew H. Beck,et al.  Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer , 2017, JAMA.

[118]  Nicolette de Keizer,et al.  Forty years of SNOMED: a literature review , 2008, BMC Medical Informatics Decis. Mak..

[119]  Nicolai Schoch,et al.  Cognitive tools pipeline for assistance of mitral valve surgery , 2016, SPIE Medical Imaging.

[120]  José L. V. Mejino,et al.  A reference ontology for biomedical informatics: the Foundational Model of Anatomy , 2003, J. Biomed. Informatics.

[121]  Nicolette de Keizer,et al.  Model Formulation: Defining and Improving Data Quality in Medical Registries: A Literature Review, Case Study, and Generic Framework , 2002, J. Am. Medical Informatics Assoc..