Multi-perspective workflow modeling for online surgical situation models

INTRODUCTION Surgical workflow management is expected to enable situation-aware adaptation and intelligent systems behavior in an integrated operating room (OR). The overall aim is to unburden the surgeon and OR staff from both manual maintenance and information seeking tasks. A major step toward intelligent systems behavior is a stable classification of the surgical situation from multiple perspectives based on performed low-level tasks. MATERIAL AND METHODS The present work proposes a method for the classification of surgical situations based on multi-perspective workflow modeling. A model network that interconnects different types of surgical process models is described. Various aspects of a surgical situation description were considered: low-level tasks, high-level tasks, patient status, and the use of medical devices. A study with sixty neurosurgical interventions was conducted to evaluate the performance of our approach and its robustness against incomplete workflow recognition input. RESULTS A correct classification rate of over 90% was measured for high-level tasks and patient status. The device usage models for navigation and neurophysiology classified over 95% of the situations correctly, whereas the ultrasound usage was more difficult to predict. Overall, the classification rate decreased with an increasing level of input distortion. DISCUSSION Autonomous adaptation of medical devices and intelligent systems behavior do not currently depend solely on low-level tasks. Instead, they require a more general type of understanding of the surgical condition. The integration of various surgical process models in a network provided a comprehensive representation of the interventions and allowed for the generation of extensive situation descriptions. CONCLUSION Multi-perspective surgical workflow modeling and online situation models will be a significant pre-requisite for reliable and intelligent systems behavior. Hence, they will contribute to a cooperative OR environment.

[1]  Nassir Navab,et al.  On-line Recognition of Surgical Activity for Monitoring in the Operating Room , 2008, AAAI.

[2]  T Neumuth,et al.  Connecting workflow management to the OR network: Design and evaluation of a bridge to enable dynamic systems behaviour , 2012, Biomedizinische Technik. Biomedical engineering.

[3]  Mokhtar Sellami,et al.  Semi-continuous HMMs with explicit state duration for unconstrained Arabic word modeling and recognition , 2008, Pattern Recognit. Lett..

[4]  J J Struijk,et al.  Segmentation of heart sound recordings by a duration-dependent hidden Markov model , 2010, Physiological measurement.

[5]  Jenny Dankelman,et al.  Discovery of high-level tasks in the operating room , 2011, J. Biomed. Informatics.

[6]  Stephen H. D. Jackson.  INT CONGR SER , 2001 .

[7]  Blake Hannaford,et al.  Generalized approach for modeling minimally invasive surgery as a stochastic process using a discrete Markov model , 2006, IEEE Transactions on Biomedical Engineering.

[8]  Wil M. P. van der Aalst,et al.  Schedule-Aware Workflow Management Systems , 2010, Trans. Petri Nets Other Model. Concurr..

[9]  Michael Kranzfelder,et al.  New technologies for information retrieval to achieve situational awareness and higher patient safety in the surgical operating room: the MRI institutional approach and review of the literature , 2011, Surgical Endoscopy.

[10]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[11]  Bor-Sen Chen,et al.  Robust power control of CDMA cellular radio systems with time-varying delays , 2010, Signal Process..

[12]  Michael W. Vannier,et al.  The operating room and the need for an IT infrastructure and standards , 2006, International Journal of Computer Assisted Radiology and Surgery.

[13]  Pierre Jannin,et al.  Surgical Phases Detection from Microscope Videos by Combining SVM and HMM , 2010, MCV.

[14]  Stefanie Speidel,et al.  Recognition of surgical skills using hidden Markov models , 2009, Medical Imaging.

[15]  L. Greenfield,et al.  Burnout among American surgeons. , 2001, Surgery.

[16]  Kevin Cleary,et al.  OR2020 workshop overview: operating room of the future , 2004, CARS.

[17]  T. Neumuth,et al.  Structured recording of intraoperative surgical workflows , 2006, SPIE Medical Imaging.

[18]  T Neumuth,et al.  Surgical workflow management schemata for cataract procedures. process model-based design and validation of workflow schemata. , 2012, Methods of information in medicine.

[19]  Nassir Navab,et al.  Statistical modeling and recognition of surgical workflow , 2012, Medical Image Anal..

[20]  Vimla L. Patel,et al.  Deviations from protocol in a complex Trauma environment: Errors or innovations? , 2011, J. Biomed. Informatics.

[21]  Peter Fu-Ming Hu,et al.  Real-Time Identification of Operating Room State from Video , 2007, AAAI.

[22]  Zang-Hee Cho,et al.  In-vivo Human Molecular Imaging Using PET-MRI Fusion Imaging Technology , 2012 .

[23]  Thomas Neumuth,et al.  Online recognition of surgical instruments by information fusion , 2012, International Journal of Computer Assisted Radiology and Surgery.

[24]  Gernot A. Fink,et al.  Markov models for offline handwriting recognition: a survey , 2009, International Journal on Document Analysis and Recognition (IJDAR).

[25]  Mathias Weske,et al.  Business Process Management: Concepts, Languages, Architectures , 2007 .

[26]  Shunzheng Yu,et al.  A hidden semi-Markov model with missing data and multiple observation sequences for mobility tracking , 2003, Signal Process..

[27]  Ping Yang,et al.  Online pattern recognition based on a generalized hidden Markov model for intraoperative vital sign monitoring , 2009 .

[28]  Vimla L. Patel,et al.  Toward automated workflow analysis and visualization in clinical environments , 2011, J. Biomed. Informatics.

[29]  Julian M Goldman,et al.  Introducing new technology into the operating room: measuring the impact on job performance and satisfaction. , 2005, Surgery.

[30]  Jeff Sutherland,et al.  Towards an Intelligent Hospital Environment: Adaptive Workflow in the OR of the Future , 2006, Proceedings of the 39th Annual Hawaii International Conference on System Sciences (HICSS'06).

[31]  Thomas Neumuth,et al.  Sensor-based surgical activity recognition in unconstrained environments , 2014, Minimally invasive therapy & allied technologies : MITAT : official journal of the Society for Minimally Invasive Therapy.

[32]  Thomas Neumuth,et al.  Intervention time prediction from surgical low-level tasks , 2013, J. Biomed. Informatics.

[33]  E. Kyriacou,et al.  Acompletely Decentralizedworkflow Management System for the Support of Emergency Telemedicine and Patient Monitoring , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[34]  Gero Strauß,et al.  Research Paper: Validation of Knowledge Acquisition for Surgical Process Models , 2009, J. Am. Medical Informatics Assoc..

[35]  Rüdiger Dillmann,et al.  Knowledge-Based Situation Interpretation for Context-Aware Augmented Reality in Dental Implant Surgery , 2010, MIAR.