Contrastive Conditioning for Assessing Disambiguation in MT: A Case Study of Distilled Bias (2021.emnlp-main)
Copied to clipboard
| Challenge: | Lexical disambiguation is a major challenge for machine translation systems . previous work focused on automatic post-hoc analysis of translations, but rules of what makes a disambiguations correct or incorrect tend to be imprecise. |
| Approach: | They propose a black-box method that uses contrastive conditioning to detect disambiguation errors. |
| Outcome: | The proposed method is scalable and reliable for disambiguation evaluations. |
Similar Papers
TEACH: A Contrastive Knowledge Adaptive Distillation Framework for Classical Chinese Understanding (2025.acl-long)
Copied to clipboard
| Challenge: | Traditional methods for processing classical Chinese segment language understanding into discrete tasks, which overlook crucial background information and reduce user engagement. |
| Approach: | They propose a framework that integrates word sense disambiguation with sentence translation to minimize hallucinations and improve semantic analysis. |
| Outcome: | The proposed framework integrates word sense disambiguation with sentence translation to minimize hallucinations and improve semantic analysis. |
As Little as Possible, as Much as Necessary: Detecting Over- and Undertranslations with Contrastive Conditioning (2022.acl-short)
Copied to clipboard
| Challenge: | Neural machine translation is susceptible to coverage errors such as the addition of superfluous target words or the omission of important source content. |
| Approach: | They propose a method for detecting Omission and addition errors with off-the-shelf translation models. |
| Outcome: | The proposed method is comparable to a supervised method that requires a custom quality estimation model. |
Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Decoding (2024.eacl-short)
Copied to clipboard
| Challenge: | Hallucinations and off-target translations remain unsolved problems in machine translation, especially for low-resource languages and massively multilingual models. |
| Approach: | They propose two methods to mitigate hallucinations and off-target translations with a modified decoding objective without retraining or external models. |
| Outcome: | The proposed methods reduce translation errors with segment-level chrF2 below 10 by 67-83% on average across 57 tested translation directions. |
Training-Free Test-Time Contrastive Learning for Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing training-free alternatives to training-based models are static or depend on external guidance. |
| Approach: | They propose a training-free adaptation framework that enables a frozen LLM to improve online by distilling supervision from its own inference experiences. |
| Outcome: | The proposed framework outperforms existing test-time adaptation methods under online evaluation. |
Fine-grained Contrastive Learning for Definition Generation (2022.aacl-main)
Copied to clipboard
| Challenge: | Recent pre-trained transformer-based definition generation models lack effective representation learning to contain full semantic components of the given word, leading to under-specific definitions. |
| Approach: | They propose a novel contrastive learning method that encourages the model to capture more detailed semantic representations from the definition sequence encoding. |
| Outcome: | The proposed method could generate more specific definitions compared with state-of-the-art models. |
Nibbling at the Hard Core of Word Sense Disambiguation (2022.acl-long)
Copied to clipboard
| Challenge: | Word Sense Disambiguation (WSD) is a task that is based on a set of pre-trained language models. |
| Approach: | They propose to use Word Sense Disambiguation to test whether systems can handle ambiguous words. |
| Outcome: | The proposed benchmarks show that seven of the most representative state-of-the-art systems make trivial errors on traditional evaluation benchmarks. |
Differentiable Data Augmentation for Contrastive Sentence Representation Learning (2022.emnlp-main)
Copied to clipboard
| Challenge: | a contrastive learning framework is used to fine-tune pre-trained language models with unlabeled sentences or labeled sentences. |
| Approach: | They propose a method that makes hard positives from unlabeled sentences . they use a prefix attached to a model to allow for differentiable data augmentation . |
| Outcome: | The proposed method yields significant improvements over existing methods under semi-supervised and supervised settings. |
DEMETR: Diagnosing Evaluation Metrics for Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | BLEU scores are based on string overlap, but they are opaque in comparison to newer learned metrics. |
| Approach: | They propose a dataset to evaluate MT evaluation metrics based on linguistic perturbations in English . they find learned metrics perform substantially better than string-based metrics . |
| Outcome: | The proposed dataset shows that learned metrics perform better than string-based metrics . the dataset contains 31K English examples that cover 35 different linguistic phenomena . |
DistillCSE: Distilled Contrastive Learning for Sentence Embeddings (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to sentence embeddings are based on contrastive learning (CL) . |
| Approach: | They propose a framework which performs contrastive learning under the self-training paradigm with knowledge distillation and propose 'Group-P shuffling strategy' and averaging logits from multiple teacher components. |
| Outcome: | The proposed framework outperforms many strong baseline methods and yields a new state-of-the-art performance. |
RoDEval: A Robust Word Sense Disambiguation Evaluation Framework for Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies rely on single-task evaluations and classification-based metrics that overlook the fundamental differences between generative LLMs and traditional classification models. |
| Approach: | They propose to use four new metrics to evaluate LLM-based word sense disambiguation (WSD) . experimental results reveal significant limitations in LLMs' WSD performance . |
| Outcome: | The proposed evaluation framework is open-source at https://github.com/DayDream405/RoDEval. |