Semantic indexing and document retrieval for personalized language modeling

The paper presents a semantic indexing and document retrieval approach for personalized language modeling to improve speech recognition accuracy for individual speakers in a lecture speech transcription task. The latent semantic indexing and paragraph vector modeling are implemented to retrieve a subset of documents from an existing background corpus relevant to the topic and speaking style of a speaker. We select a subset of text documents semantically similar to the output hypotheses from recognized speech segments in the first decoding stage. After that, a small user topic-specific language model is created from the relevant documents, interpolated with the background model, adapted to the current topic and applied during the second decoding stage. Experimental results performed for ten speakers from the database of the Slovak TEDx talks show an improvement in word error rate up to 3.03% relatively on average.