Expert Evaluation of a Spoken Dialogue System in a Clinical Operating Room

With the emergence of new technologies, the surgical working environment becomes increasingly complex and comprises many medical devices which have to be monitored and controlled. With the aim of improving productivity and reducing the workload for the operating staff, we have developed an Intelligent Digital Assistant for Clinical Operating Rooms (IDACO) which allows the surgeon to control the operating room using natural spoken language. As speech is the modality used by the surgeon to communicate with their staff, using it to control the technical devices does not pose an additional mental burden. Therefore, we claim that the surgical environment presents a potential field of application for Spoken Dialogue Systems. In this work, we present the design and implementation of IDACO as well as the evaluation in an experimental set-up by specialists in the field of minimally invasive surgery. Our expert evaluation yields promising results and allows to conclude that clinical operating rooms are indeed an expedient area of application for Spoken Dialogue Systems.

[1]  D B Roe,et al.  A voice-controlled network for universal control of devices in the OR , 2000, Minimally invasive therapy & allied technologies : MITAT : official journal of the Society for Minimally Invasive Therapy.

[2]  Michael Kranzfelder,et al.  New technologies for information retrieval to achieve situational awareness and higher patient safety in the surgical operating room: the MRI institutional approach and review of the literature , 2011, Surgical Endoscopy.

[3]  Michael Kranzfelder,et al.  Reliability of sensor-based real-time workflow recognition in laparoscopic cholecystectomy , 2014, International Journal of Computer Assisted Radiology and Surgery.

[4]  H. Feußner,et al.  Technische und digitale Weiterentwicklung in der laparoskopischen/offenen Chirurgie , 2014, Der Chirurg.

[5]  Jeff Sutherland,et al.  Towards an Intelligent Hospital Environment: Adaptive Workflow in the OR of the Future , 2006, Proceedings of the 39th Annual Hawaii International Conference on System Sciences (HICSS'06).

[6]  Wolfgang Minker,et al.  Managing adaptive spoken dialogue for Intelligent Environments , 2014, J. Ambient Intell. Smart Environ..

[7]  H. Feußner The Operating Room of the Future: A View from Europe , 2003, Seminars in laparoscopic surgery.

[8]  Timothy W. Finin,et al.  A Pervasive Computing System for the Operating Room of the Future , 2007, Mob. Networks Appl..

[9]  Juliana Miehle,et al.  The next step: intelligent digital assistance for clinical operating rooms , 2017, Innovative surgical sciences.

[10]  H. Feußner,et al.  Minimalinvasive Chirurgie und „robotic surgery“: Chirurgie 4.0? , 2016, Der Chirurg.

[11]  Nadine Gerstenlauer Controlling an intelligent operating room with spoken language , 2017 .

[12]  Nicolas Padoy,et al.  Workflow and Activity Modeling for Monitoring Surgical Procedures. (Modélisation des Activités Chirurgicales et de leur Déroulement pour la Reconnaissance des Etapes Opératoires) , 2010 .

[13]  H. Feußner,et al.  Real-time instrument detection in minimally invasive surgery using radiofrequency identification technology. , 2013, The Journal of surgical research.

[14]  Alexander Schmitt,et al.  OwlSpeak - adaptive spoken dialogue within Intelligent Environments , 2010, 2010 8th IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops).

[15]  Joanne Pransky,et al.  An intelligent operating room of the future – an interview with the University of California Los Angeles Medical Center , 2001 .

[16]  Frank van Harmelen,et al.  Web Ontology Language: OWL , 2004, Handbook on Ontologies.

[17]  Rüdiger Dillmann,et al.  Knowledge-Driven Formalization of Laparoscopic Surgeries for Rule-Based Intraoperative Context-Aware Assistance , 2014, IPCAI.

[18]  Mike Potel,et al.  MVP: Model-View-Presenter The Taligent Programming Model for C++ and Java , 1996 .

[19]  J. Donohue Laparoscopic cholecystectomy. , 1992, Surgery.