Measuring MT Adequacy Using Latent Semantic Analysis

Translation adequacy is defined as the amount of semantic content from the source language document that is conveyed in the target language document. As such, it is more difficult to measure than intelligibility since semantic content must be measured in two documents and then compared. Latent Semantic Analysis is a content measurement technique used in language learner evaluation that exhibits characteristics attractive for re-use in machine translation evaluation (MTE). This experiment, which is a series of applications of the LSA algorithm in various configurations, demonstrates its usefulness as an MTE metric for adequacy. In addition, this experiment lays the groundwork for using LSA as a method to measure the accuracy of a translation without reliance on reference translations.

[1]  Peter W. Foltz,et al.  Supporting Content-Based Feedback in On-Line Writing Evaluation with LSA , 2000, Interact. Learn. Environ..

[2]  Sui Dan-ni.  Latent Semantic Analysis for Text-Based Research , 2005 .

[3]  John S. White,et al.  The ARPA MT Evaluation Methodologies: Evolution, Lessons, and Future Approaches , 1994, AMTA.

[4]  P. Foltz,et al.  Content-based feedback 1 Supporting content-based feedback in online writing evaluation with LSA , 2000 .

[5]  Bob Rehder,et al.  Using latent semantic analysis to assess knowledge: Some technical considerations , 1998 .

[6]  Peter W. Foltz,et al.  The intelligent essay assessor: Applications to educational technology , 1999 .

[7]  Martin Chodorow,et al.  Computer Analysis of Essay Content for Automated Score Prediction , 1998 .

[8]  Arthur C. Graesser,et al.  Using Latent Semantic Analysis to Evaluate the Contributions of Students in AutoTutor , 2000, Interact. Learn. Environ..

[9]  T. Landauer,et al.  Indexing by Latent Semantic Analysis , 1990 .

[10]  Bob Rehder,et al.  How Well Can Passage Meaning be Derived without Using Word Order? A Comparison of Latent Semantic Analysis and Humans , 1997 .

[11]  Thomas K. Landauer,et al.  Simulating Text Understanding for Educational Applications with Latent Semantic Analysis: Introduction to LSA , 2000, Interact. Learn. Environ..

[12]  Peter W. Foltz,et al.  Learning Human-like Knowledge by Singular Value Decomposition: A Progress Report , 1997, NIPS.

[13]  Michael W. Berry,et al.  GTP (General Text Parser) Software for Text Mining , 2003 .

[14]  Florence Reeder,et al.  Granularity in MT evaluation , 2003, MTSUMMIT.

[15]  A. Graesser,et al.  Improving an intelligent tutor ’ s comprehension of students with Latent Semantic Analysis ∗ , 1999 .

[16]  Susan T. Dumais,et al.  Improving the retrieval of information from external sources , 1991 .

[17]  Richard A. Harshman,et al.  Information retrieval using a singular value decomposition model of latent semantic structure , 1988, SIGIR '88.

[18]  Peter W. Foltz,et al.  Reasoning from Multiple Texts: An Automatic Analysis of Readers? Situation Models , 1996 .

[19]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[20]  Peter W. Foltz,et al.  The Measurement of Textual Coherence with Latent Semantic Analysis. , 1998 .

[21]  Peter W. Foltz,et al.  An introduction to latent semantic analysis , 1998 .

[22]  Martin Chodorow,et al.  Automated Essay Scoring for Nonnative English Speakers , 1999 .

[23]  Rodolfo Soto,et al.  Learning and performing by exploration: label quality measured by latent semantic analysis , 1999, CHI '99.