Improving on-line handwritten recognition in interactive machine translation

On-line handwriting text recognition (HTR) could be used as a more natural way of interaction in many interactive applications. However, current HTR technology is far from developing error-free systems and, consequently, its use in many applications is limited. Despite this, there are many scenarios, as in the correction of the errors of fully-automatic systems using HTR in a post-editing step, in which the information from the specific task allows to constrain the search and therefore to improve the HTR accuracy. For example, in machine translation (MT), the on-line HTR system can also be used to correct translation errors. The HTR can take advantage of information from the translation problem such as the source sentence that is translated, the portion of the translated sentence that has been supervised by the human, or the translation error to be amended. Empirical experimentation suggests that this is a valuable information to improve the robustness of the on-line HTR system achieving remarkable results. Graphical abstractThis work presents an e-pen enabled system where handwriting is used to amend the errors of a machine translation system. Handwriting recognition is performed in such a way that the contextual information (source, prefix, translation, and error) is integrated to improve the final recognition accuracy.Display Omitted HighlightsWe present a specific on-line HTR system for editing machine translation (MT) output.We leverage information from different sources in MT to constrain the HTR search.All the proposed systems outperform the baseline.The use of information from the translation models achieves remarkable results.Finally, we propose a system to amend HTR errors with a 75% typing effort reduction.