Exploring Supervised and Unsupervised Rewards in Machine Translation (2021.eacl-main)
Copied to clipboard
| Challenge: | Autoregressive sequence-to-sequence (seq2sequ) neural architectures have become the de facto approach in Machine Translation (MT). |
| Approach: | They propose to make models less reliant on cross-entropy loss and evaluation metrics . they propose an entropicity-regularised RL method that explores the action space . |
| Outcome: | The proposed method exploits the action space and unsupervised reward function to balance between exploration and exploitation. |
Similar Papers
Breaking Consensus Bias: Unsupervised Reinforcement Learning for Machine Translation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing RL approaches for MT face fixed references or the production of homogeneous references leading to mode collapse in unsupervised settings. |
| Approach: | They propose an Entropy-Driven Unsupervised RL framework for machine translation that leverages entropy for supervision construction and self-evolution. |
| Outcome: | The proposed framework outperforms supervised and unsupervised baselines in multiple language pairs. |
SURF: Semantic-level Unsupervised Reward Function for Machine Translation (2022.naacl-main)
Copied to clipboard
| Challenge: | Reinforcement Learning (RL) is dependent on the reward formulation due to the intrinsic difficulty of the task in the high-dimensional discrete action space and the sparseness of the standard reward functions. |
| Approach: | They propose a maximally dense semantic-level unsupervised reward function which mimics human evaluation by considering both sentence fluency and semantic similarity. |
| Outcome: | The proposed reward outperforms the standard sparse reward by 2% on average for in- and out-of-domain settings. |
MT-R1-Zero: Advancing LLM-based Machine Translation via R1-Zero-like Reinforcement Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large-scale reinforcement learning (RL) methods have proven effective in enhancing the reasoning abilities of large language models. |
| Approach: | They propose an open-source adaptation of the R1-Zero RL framework for machine translation (MT) their code is available at https://github.com/fzp0424/MT-R1-zero. |
| Outcome: | The proposed framework surpasses towerinstruct-7B-v0.2 on the english-chinese benchmark by 1.26 points. |
Fine-Grained Reward Optimization for Machine Translation using Error Severity Mappings (2026.tacl-1)
Copied to clipboard
Miguel Moura Ramos, Tomás Almeida, Daniel Vareta, Filipe Azevedo, Sweta Agrawal, Patrick Fernandes, André F. T. Martins
| Challenge: | Reinforcement learning (RL) is an effective and robust method for training neural machine translation systems. |
| Approach: | They propose a method that leverages fine-grained, token-level quality assessments . they use a state-of-the-art quality estimation system as their token- level reward model . |
| Outcome: | The proposed approach leverages fine-grained, token-level quality assessments along with error severity levels to improve translation quality. |
Enhancing Reinforcement Learning with Dense Rewards from Language Model Critic (2024.emnlp-main)
Copied to clipboard
| Challenge: | Reinforcement learning (RL) can align language models with non-differentiable reward signals, such as human preferences, but the sparsity of these signals can lead to inefficient and unstable learning. |
| Approach: | They propose a framework that utilizes the critique capability of Large Language Models to produce intermediate-step rewards during RL training. |
| Outcome: | The proposed framework improves sample efficiency and the overall performance of the policy model, supported by both automatic and human evaluation. |
Countering Reward Over-Optimization in LLM with Demonstration-Guided Reinforcement Learning (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches address ROO by adding KL regularization, requiring computationally expensive hyperparameter tuning. |
| Approach: | They propose a reinforcement learning approach that leverages human demonstrations and a reward model to recalibrate the reward objective. |
| Outcome: | The proposed approach achieves comparable performance to carefully tuned baselines while mitigating ROO in three RL language tasks. |
A Study of Reinforcement Learning for Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Recent studies have shown that reinforcement learning (RL) is an effective approach for improving the performance of neural machine translation systems. |
| Approach: | They propose to leverage reinforcement learning to boost the performance of NMT systems trained with monolingual data. |
| Outcome: | The proposed method achieves competitive results on translation tasks in English-German, Chinese-English and English-English systems. |
RED: Unleashing Token-Level Rewards from Holistic Feedback via Reward Redistribution (2025.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results demonstrate the superiority of our approach to aligning large language models with human preferences. |
| Approach: | They propose a method that evaluates and assigns specific credit to each token using an off-the-shelf reward model. |
| Outcome: | The proposed method evaluates and assigns specific credit to each token using an off-the-shelf reward model. |
Reinforcement Learning with Large Action Spaces for Neural Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Recent work has argued that the gains produced by Reinforcement learning are mostly due to promoting tokens that have already received a fairly high probability in pre-training. |
| Approach: | They hypothesize that the large action space is a main obstacle to RL’s effectiveness in MT by reducing the size of the vocabulary without changing the vocabulary. |
| Outcome: | The proposed method improves by 1.5 BLEU points on average. |
RIVAL: Reinforcement Learning with Iterative and Adversarial Optimization for Machine Translation (2025.findings-emnlp)
Copied to clipboard
Tianjiao Li, Mengran Yu, Chenyu Shi, Yanjun Zhao, Xiaojing Liu, Qi Zhang, Xuanjing Huang, Qiang Zhang, Jiayin Wang
| Challenge: | Using reinforcement learning from human feedback, large language models perform poorly when applied to colloquial subtitle translation tasks. |
| Approach: | They propose an adversarial training framework that iteratively updates the offline reward model and the online LLM to improve training outcomes. |
| Outcome: | The proposed training framework significantly improves upon translation baselines. |