Neural spelling correction: translating incorrect sentences to correct sentences for multimedia

The aim of a spelling correction task is to detect spelling errors and automatically correct them. In this paper we aim to perform the Korean spelling correction task from a machine translation perspective, allowing it to overcome the limitations of cost, time and data. Based on a sequence to sequence model, the model aligns its source sentence with an ‘error filled sentence’ and its target sentence aligned to the correct counter part. Thus, ‘translating’ the error sentence to a correct sentence. For this research, we have also proposed three new data generation methods allowing the creation of multiple spelling correction parallel corpora from just a single monolingual corpus. Additionally, we discovered that applying the Copy Mechanism not only resolves the problem of overcorrection but even improves it. For this paper, we evaluated our model upon these aspects: Performance comparisons to other models and evaluation on overcorrection. The results show the proposed model to even out-perform other systems currently in commercial use.

[1]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[2]  Hae-Chang Rim,et al.  Automatic Spelling Correction Rule Extraction and Application for Spoken-Style Korean Text , 2007, Sixth International Conference on Advanced Language Processing and Web Information Technology (ALPIT 2007).

[3]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[4]  Yves Schabes.  Exact Generalization of Finite-State Transductions : Application to Grapheme-to-Phoneme Transcription , 1995 .

[5]  Jung-Hun Lee,et al.  The Utilization of Local Document Information to Improve Statistical Context-Sensitive Spelling Error Correction , 2017 .

[6]  Huaiyu Zhu.  On Information and Sufficiency , 1997 .

[7]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[8]  Pankoo Kim,et al.  Word2Vec based spelling correction method of Twitter message , 2019, SAC.

[9]  Rico Sennrich,et al.  Neural Machine Translation of Rare Words with Subword Units , 2015, ACL.

[10]  Hyuk-Chul Kwon,et al.  Stochastic Korean Word-Spacing with Smoothing Using Korean Spelling Checker , 2004, Int. J. Comput. Process. Orient. Lang..

[11]  Pierre Isabelle,et al.  Proceedings of the 40th Annual Meeting on Association for Computational Linguistics , 2002, ACL 2002.

[12]  Shuvendu Roy,et al.  Unsupervised Context-Sensitive Bangla Spelling Correction with Character N-gram , 2019, 2019 22nd International Conference on Computer and Information Technology (ICCIT).

[13]  Alexander M. Rush,et al.  OpenNMT: Open-Source Toolkit for Neural Machine Translation , 2017, ACL.

[14]  Li Yujian,et al.  A Normalized Levenshtein Distance Metric , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Yann Dauphin,et al.  Convolutional Sequence to Sequence Learning , 2017, ICML.

[16]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[17]  Jung-Hun Lee,et al.  Improved Statistical Language Model for Context-sensitive Spelling Error Candidates , 2017 .

[18]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[19]  Kichun Nam,et al.  Korean Spelling Development and Linguistic Patterns , 2015 .

[20]  Maria da Graça Campos Pimentel,et al.  Fast Word Recognition for Noise channel-based Models in Scenarios with Noise Specific Domain Knowledge , 2017, CIKM.

[21]  Daniel Povey,et al.  The Kaldi Speech Recognition Toolkit , 2011 .

[22]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[23]  Hagen Soltau,et al.  Neural Speech Recognizer: Acoustic-to-Word LSTM Model for Large Vocabulary Speech Recognition , 2016, INTERSPEECH.

[24]  Sanjeev Khudanpur,et al.  Semi-Supervised Training of Acoustic Models Using Lattice-Free MMI , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[25]  Quoc V. Le,et al.  SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition , 2019, INTERSPEECH.

[26]  Minho Kim,et al.  Context-Sensitive Spelling Error Correction Using Inter-Word Semantic Relation Analysis , 2014, 2014 International Conference on Information Science & Applications (ICISA).

[27]  Hang Li,et al.  “ Tony ” DNN Embedding for “ Tony ” Selective Read for “ Tony ” ( a ) Attention-based Encoder-Decoder ( RNNSearch ) ( c ) State Update s 4 SourceVocabulary Softmax Prob , 2016 .

[28]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[29]  Colin Raffel,et al.  Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition , 2019, ICML.

[30]  Walter Daelemans,et al.  Unsupervised Context-Sensitive Spelling Correction of Clinical Free-Text with Word and Character N-Gram Embeddings , 2017, BioNLP.