An Exploration of Placeholding in Neural Machine Translation

Phrase-based machine translation provides the system developer with controls that enable fine-grained control over machine translation output. One approach to provide similar control in neural machine translation is placeholding (herein called masking), which replaces input tokens with masks which are replaced with the original input text in post-processing. But is this a good idea? We undertake an exploration of masking in French–English and Japanese–English using Transformer architectures. We attempt to quantify whether (and where) masking is necessary with analysis of a baseline system, and then explore numerous parameterization of masking, including post-processing techniques for replacing the masks. Our analysis shows this to be a thorny matter; masks solve some problems but are not perfectly translated themselves.