Metamorphic Relations for Data Validation: A Case Study of Translated Text Messages

In conventional metamorphic testing, metamorphic relations (MRs) are identified as necessary properties of a computer program's intended functionality, whereby violations of MRs reveal faults in the program—under the assumption that the source and follow-up inputs (test cases used in metamorphic testing) are valid. In the present study, the authors argue that MRs can also be used to validate and assess the quality of the program's input data—under the assumption that the source or follow-up inputs can be inappropriately generated. Using this new perspective, a case study in the natural language processing domain is used to explore the different types of text messages that are difficult to interpret by (Chinese-English) machine translation. A total of 46,180 short user comments on Personal Tailor (a 2013 Chinese film), collected from Douban (a popular Chinese social media platform), has been used as the primary dataset of this study, and the analysis of results demonstrates that the proposed MR-based data validation method is useful for the automatic identification of poorly translated text messages.