Automatic Machine Translation Evaluation using Source Language Inputs and Cross-lingual Language Model (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for machine translation evaluation use source sentences as pseudo references instead of word symbols. |
| Approach: | They propose an automatic machine translation evaluation method that uses source sentences as pseudo references instead of source sentences. |
| Outcome: | The proposed method achieves higher correlation with human judgments than baseline evaluation method that uses only hypothesis and reference sentences. |
Similar Papers
Lost in the Source Language: How Large Language Models Evaluate the Quality of Machine Translation (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that Large Language Models (LLMs) can be used as translation evaluators. |
| Approach: | They propose to use both coarse-grained and fine-grounded prompts to discern the utility of source versus reference data in machine translation evaluation tasks. |
| Outcome: | The proposed model can be used to evaluate translations in multiple languages. |
Improving Multilingual Neural Machine Translation with Auxiliary Source Languages (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work has shown that translating from multiple source languages improves translation quality. |
| Approach: | They propose to exploit multiple source sentences from auxiliary languages to improve multilingual translation in a more common scenario by using synthetic multi-source corpora. |
| Outcome: | Extensive experiments on Chinese/English-Japanese and a large-scale multilingual translation benchmark show that the proposed model outperforms the baseline model significantly by +4.0 BLEU. |
What do Large Language Models Need for Machine Translation Evaluation? (2024.emnlp-main)
Copied to clipboard
Shenbin Qian, Archchana Sindhujan, Minnie Kabra, Diptesh Kanojia, Constantin Orasan, Tharindu Ranasinghe, Fred Blain
| Challenge: | Existing research shows that large language models can perform better in machine translation tasks. |
| Approach: | They propose to use large language models for machine translation evaluations . authors explore what translation information is needed for LLMs to evaluate MT quality . |
| Outcome: | The proposed model performs comparable to fine-tuned multilingual pre-trained models. |
Putting Evaluation in Context: Contextual Embeddings Improve Machine Translation Evaluation (P19-1)
Copied to clipboard
| Challenge: | Existing evaluation metrics are limited and can be easily portable to new languages. |
| Approach: | They propose a simple unsupervised metric and additional supervised metrics which rely on contextual word embeddings to encode the translation and reference sentences. |
| Outcome: | The proposed model outperforms existing metrics on the WMT 2017 dataset and is more accurate than existing models. |
Metric for Automatic Machine Translation Evaluation based on Universal Sentence Representations (N18-4)
Copied to clipboard
| Challenge: | Sentence representations can capture information that cannot be captured by local features based on character or word Ngrams. |
| Approach: | They propose a supervised regression model using universal sentence representations capable of capturing information that cannot be captured by local features based on character or word Ngrams. |
| Outcome: | The proposed model achieves state-of-the-art performance with only sentence representation features . |
Quality Scoring of Source Words in Neural Translation Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent approaches to improving word-level quality scores on input source sentences require training special word-scoring models or require repeated invocation of the translation model. |
| Approach: | They propose to reason how well each word is explained by the target sentence as against the source language model and use it to translate into an unfamiliar target language. |
| Outcome: | The proposed method provides up to five points higher F1 scores and is significantly faster than the state of the art methods on three language pairs. |
XLM-D: Decorate Cross-lingual Pre-training Model as Non-Autoregressive Neural Machine Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing pre-training language models have been successful in natural language understanding and autoregressive generation tasks, but non-autoregressive models have not been sufficiently successful. |
| Approach: | They propose a pre-trained masked language model (MLM) and a non-autoregressive generation model with a lightweight decorator. |
| Outcome: | The proposed model outperforms the previous mask-predict model on translation datasets by 19.9x. |
Evaluating Language Translation Models by Playing Telephone (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing language models are inadequate for evaluating machine translation systems . current evaluation methods are costly and require specialized expertise to prepare and score gold standard translations . |
| Approach: | They propose an unsupervised method to generate training data for translation evaluation by repeated rounds of translation between source and target languages. |
| Outcome: | The proposed method outperforms a popular translation evaluation system on two tasks . human annotation is costly and requires specialized expertise to prepare and score gold standard translations . |
Context-Interactive Pre-Training for Document Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Document machine translation typically suffers from a lack of document-level bilingual data. |
| Approach: | They propose a document machine translation model that incorporates contextual information into the training signals by capturing cross-sentence dependency within the target document and cross sentence translation to make better use of contextual information. |
| Outcome: | The proposed model outperforms baselines on three benchmark datasets and significantly outperformed previous approaches. |
Multi-Source Text Classification for Multilingual Sentence Encoder with Machine Translation (2024.naacl-srw)
Copied to clipboard
| Challenge: | Pre-trained multilingual sentence encoders suffer from performance degradation for non-English languages. |
| Approach: | They propose a method of machine translating a source sentence into English and then inputting it together with the source sentence in a multi-source manner. |
| Outcome: | The proposed method improves the performance of pre-trained multilingual sentence encoders in Japanese on sentiment analysis and topic classification tasks. |