Online recognition of surgical instruments by information fusion

PurposeAutomatic online recognition of surgical instruments is required to monitor instrument use for surgical process modeling. A system was developed and tested using available technologies.MethodsA recognition system was developed using RFID technology to identify surgical activities. Information fusion for online recognition of surgical process models was conceived as a layer model to abstract information from specific sensor technologies. Redundant, complementary, and cooperative sensor signal fusion was used in the layer model to increase the surgical instrument recognition rate. Several different information fusion strategies were evaluated for situation recognition abilities in a mock-up environment based on simulations of surgical processes.ResultsThis information fusion system was able to reliably detect, identify, and localize surgical instruments in an interventional suite. A combination of information fusion strategies was able to achieve a correct classification rate of 97% and was as effective as observer-based acquisition methods.ConclusionDifferent information fusion strategies for the recognition of surgical instruments were evaluated, showing that redundant, complementary, and cooperative information fusion is feasible for recognition of surgical work steps. A combination of sensor- and observer-based modeling strategies provides the most robust solution for surgical process models.

[1]  Matthijs Oudkerk,et al.  PACS storage requirements - influence of changes in imaging modalities , 2005 .

[2]  Ren C. Luo,et al.  Multisensor fusion and integration: approaches, applications, and future research directions , 2002 .

[3]  Michael W. Vannier,et al.  The operating room and the need for an IT infrastructure and standards , 2006, International Journal of Computer Assisted Radiology and Surgery.

[4]  Kevin Cleary,et al.  OR 2020 workshop report: Operating room of the future , 2005 .

[5]  Nassir Navab,et al.  Workflow mining for visualization and analysis of surgeries , 2008, International Journal of Computer Assisted Radiology and Surgery.

[6]  Hugh F. Durrant-Whyte,et al.  Sensor Models and Multisensor Integration , 1988, Int. J. Robotics Res..

[7]  Gregory D. Hager,et al.  Towards automatic skill evaluation: detection and segmentation of robot-assisted surgical motions. , 2006 .

[8]  Gregory D. Hager,et al.  Data-Derived Models for Segmentation with Application to Surgical Assessment and Training , 2009, MICCAI.

[9]  R. Van der Togt,et al.  Electromagnetic interference from radio frequency identification inducing potentially hazardous incidents in critical care medical equipment. , 2008, JAMA.

[10]  Thomas Neumuth,et al.  Assessment of technical needs for surgical equipment by surgical process models , 2009, Minimally invasive therapy & allied technologies : MITAT : official journal of the Society for Minimally Invasive Therapy.

[11]  Pierre Jannin,et al.  Surgical models for computer-assisted neurosurgery , 2007, NeuroImage.

[12]  Yan Xiao,et al.  An Algorithm for Processing Vital Sign Monitoring Data to Remotely Identify Operating Room Occupancy in Real-Time , 2005, Anesthesia and analgesia.

[13]  James Llinas,et al.  Revisiting the JDL Data Fusion Model II , 2004 .

[14]  A. Darzi,et al.  Skills assessment of surgeons. , 2002, Surgery.

[15]  Mieczyslaw M. Kokar,et al.  Formalizing classes of information fusion systems , 2004, Inf. Fusion.

[16]  Thomas Neumuth,et al.  Analysis of surgical intervention populations using generic surgical process models , 2010, International Journal of Computer Assisted Radiology and Surgery.

[17]  Ning Xiong,et al.  Multi-sensor management for information fusion: issues and approaches , 2002, Inf. Fusion.

[18]  Nassir Navab,et al.  Motif Discovery in OR Sensor Data with Application to Surgical Workflow Analysis and Activity Detection , 2009 .

[19]  T Neumuth,et al.  Impact quantification of the daVinci telemanipulator system on surgical workflow using resource impact profiles , 2011, The international journal of medical robotics + computer assisted surgery : MRCAS.

[20]  Pierre Jannin,et al.  Automatic Phases Recognition in Pituitary Surgeries by Microscope Images Classification , 2010, IPCAI.

[21]  L. MacKenzie,et al.  Hierarchical decomposition of laparoscopic surgery: a human factors approach to investigating the operating room environment , 2001, Minimally invasive therapy & allied technologies : MITAT : official journal of the Society for Minimally Invasive Therapy.

[22]  T. Neumuth,et al.  Title Validation of Knowledge Acquisition for Surgical Process Models Authors , 2009 .

[23]  Stefanie Speidel,et al.  Estimating Similarity of Surgical Situations with Case-Retrieval-Nets , 2009, MMVR.

[24]  Jenny Dankelman,et al.  Discovery of high-level tasks in the operating room , 2011, J. Biomed. Informatics.

[25]  Guang-Zhong Yang,et al.  Eye-Gaze Driven Surgical Workflow Segmentation , 2007, MICCAI.

[26]  James Llinas,et al.  An introduction to multisensor data fusion , 1997, Proc. IEEE.

[27]  Blake Hannaford,et al.  Markov modeling of minimally invasive surgery based on tool/tissue interaction and force/torque signatures for evaluating surgical skills , 2001, IEEE Transactions on Biomedical Engineering.

[28]  P. Jannin,et al.  Model of Surgical Procedures for Multimodal Image-Guided Neurosurgery , 2003, Computer aided surgery : official journal of the International Society for Computer Aided Surgery.

[29]  Nassir Navab,et al.  Statistical modeling and recognition of surgical workflow , 2012, Medical Image Anal..

[30]  Peter Fu-Ming Hu,et al.  Real-Time Identification of Operating Room State from Video , 2007, AAAI.