Automatically Detecting Likely Edits in Clinical Notes Created Using Automatic Speech Recognition

The use of automatic speech recognition (ASR) to create clinical notes has the potential to reduce costs associated with note creation for electronic medical records, but at current system accuracy levels, post-editing by practitioners is needed to ensure note quality. Aiming to reduce the time required to edit ASR transcripts, this paper investigates novel methods for automatic detection of edit regions within the transcripts, including both putative ASR errors but also regions that are targets for cleanup or rephrasing. We create detection models using logistic regression and conditional random field models, exploring a variety of text-based features that consider the structure of clinical notes and exploit the medical context. Different medical text resources are used to improve feature extraction. Experimental results on a large corpus of practitioner-edited clinical notes show that 67% of sentence-level edits and 45% of word-level edits can be detected with a false detection rate of 15%.

[1]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[2]  Harald Trost,et al.  Using Domain Knowledge about Medications to Correct Recognition Errors in Medical Report Creation , 2010, Louhi@NAACL-HLT.

[3]  Kaisheng Yao,et al.  Estimating confidence scores on ASR results using recurrent neural networks , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[4]  Yannick Estève,et al.  Word embeddings combination and neural networks for robustness in ASR error detection , 2015, 2015 23rd European Signal Processing Conference (EUSIPCO).

[5]  M. Stella Atkins,et al.  Improving the Utility of Speech Recognition Through Error Detection , 2008, Journal of Digital Imaging.

[6]  Robert L. Mercer,et al.  Class-Based n-gram Models of Natural Language , 1992, CL.

[7]  Enrico W. Coiera,et al.  Risks and benefits of speech recognition for clinical documentation: a systematic review , 2016, J. Am. Medical Informatics Assoc..

[8]  Andreas Stolcke,et al.  Enriching speech recognition with automatic detection of sentence boundaries and disfluencies , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[9]  Atsunori Ogawa,et al.  ASR error detection and recognition rate estimation using deep bidirectional recurrent neural networks , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[10]  William D. Lewis,et al.  Intelligent Selection of Language Model Training Data , 2010, ACL.

[11]  Frédéric Béchet,et al.  ASR error segment localization for spoken recovery strategy , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[12]  Alfons Juan-Císcar,et al.  ASR Confidence Estimation with Speaker-Adapted Recurrent Neural Networks , 2016, INTERSPEECH.

[13]  Gökhan Tür,et al.  Automatic disfluency removal for improving spoken language translation , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[14]  Mari Ostendorf,et al.  Effective data-driven feature learning for detecting name errors in automatic speech recognition , 2014, 2014 IEEE Spoken Language Technology Workshop (SLT).

[15]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[16]  Hermann Ney,et al.  Improved backing-off for M-gram language modeling , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[17]  Isabel Trancoso,et al.  Improving ASR error detection with non-decoder based features , 2010, INTERSPEECH.

[18]  Kimberly D. Voll A Hybrid Approach to Improving Automatic Speech Recognition Via NLP , 2007, Canadian Conference on AI.