SURF: Semantic-level Unsupervised Reward Function for Machine Translation (2022.naacl-main)
Copied to clipboard
| Challenge: | Reinforcement Learning (RL) is dependent on the reward formulation due to the intrinsic difficulty of the task in the high-dimensional discrete action space and the sparseness of the standard reward functions. |
| Approach: | They propose a maximally dense semantic-level unsupervised reward function which mimics human evaluation by considering both sentence fluency and semantic similarity. |
| Outcome: | The proposed reward outperforms the standard sparse reward by 2% on average for in- and out-of-domain settings. |
Similar Papers
Exploring Supervised and Unsupervised Rewards in Machine Translation (2021.eacl-main)
Copied to clipboard
| Challenge: | Autoregressive sequence-to-sequence (seq2sequ) neural architectures have become the de facto approach in Machine Translation (MT). |
| Approach: | They propose to make models less reliant on cross-entropy loss and evaluation metrics . they propose an entropicity-regularised RL method that explores the action space . |
| Outcome: | The proposed method exploits the action space and unsupervised reward function to balance between exploration and exploitation. |
MT-R1-Zero: Advancing LLM-based Machine Translation via R1-Zero-like Reinforcement Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large-scale reinforcement learning (RL) methods have proven effective in enhancing the reasoning abilities of large language models. |
| Approach: | They propose an open-source adaptation of the R1-Zero RL framework for machine translation (MT) their code is available at https://github.com/fzp0424/MT-R1-zero. |
| Outcome: | The proposed framework surpasses towerinstruct-7B-v0.2 on the english-chinese benchmark by 1.26 points. |
Reinforcement Learning with Large Action Spaces for Neural Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Recent work has argued that the gains produced by Reinforcement learning are mostly due to promoting tokens that have already received a fairly high probability in pre-training. |
| Approach: | They hypothesize that the large action space is a main obstacle to RL’s effectiveness in MT by reducing the size of the vocabulary without changing the vocabulary. |
| Outcome: | The proposed method improves by 1.5 BLEU points on average. |
Breaking Consensus Bias: Unsupervised Reinforcement Learning for Machine Translation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing RL approaches for MT face fixed references or the production of homogeneous references leading to mode collapse in unsupervised settings. |
| Approach: | They propose an Entropy-Driven Unsupervised RL framework for machine translation that leverages entropy for supervision construction and self-evolution. |
| Outcome: | The proposed framework outperforms supervised and unsupervised baselines in multiple language pairs. |
SSR-Zero: Simple Self-Rewarding Reinforcement Learning for Machine Translation (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated remarkable capabilities in machine translation, but most MT-specific LLMs rely heavily on external supervision during training. |
| Approach: | They propose a reinforcement learning framework for machine translation that is reference-free and relies solely on self-judging rewards. |
| Outcome: | The proposed framework outperforms existing LLMs and larger general LLM models on English Chinese translation benchmarks and performs competitively with leading closed-source systems. |
A Study of Reinforcement Learning for Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Recent studies have shown that reinforcement learning (RL) is an effective approach for improving the performance of neural machine translation systems. |
| Approach: | They propose to leverage reinforcement learning to boost the performance of NMT systems trained with monolingual data. |
| Outcome: | The proposed method achieves competitive results on translation tasks in English-German, Chinese-English and English-English systems. |
Fine-Grained Reward Optimization for Machine Translation using Error Severity Mappings (2026.tacl-1)
Copied to clipboard
Miguel Moura Ramos, Tomás Almeida, Daniel Vareta, Filipe Azevedo, Sweta Agrawal, Patrick Fernandes, André F. T. Martins
| Challenge: | Reinforcement learning (RL) is an effective and robust method for training neural machine translation systems. |
| Approach: | They propose a method that leverages fine-grained, token-level quality assessments . they use a state-of-the-art quality estimation system as their token- level reward model . |
| Outcome: | The proposed approach leverages fine-grained, token-level quality assessments along with error severity levels to improve translation quality. |
Enhancing Reinforcement Learning with Label-Sensitive Reward for Natural Language Understanding (2024.acl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have yielded remarkable performance, but objective mismatch issues hinder RLHF learning. |
| Approach: | They propose a Reinforcement Learning framework enhanced with Label-sensitive reward to enhance LLMs' alignment and generation capabilities. |
| Outcome: | The proposed framework improves performance on five diverse models across eight tasks. |
From General Reward to Targeted Reward: Improving Open-ended Long-context Generation Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Current research on long-form context in Large Language Models (LLMs) focuses on understanding of long-contexts, but the open-ended Long Text Generation (Open-LTG) remains underexplored. |
| Approach: | They propose a method that uses data synthesis and a reward signal to enhance model performance. |
| Outcome: | The proposed method outperforms GPT-4-Turbo and improves performance by 20% on the Open-LTG task. |
VEG: Verbal 𝜖-greedy for Semantic Exploration in Multi-Turn RL Agents (2026.acl-industry)
Copied to clipboard
Yongchang Hao, Jie Hao, Yongsheng Mei, Ze Ye, Junyi Chai, Bin Guo, Benjamin Z. Yao, Chenlei Guo, Lili Mou
| Challenge: | Standard RL approaches suffer from reward sparsity and mode-seeking behavior . lack of diversity hinders exploration necessary for optimal learning . |
| Approach: | They propose a framework that leverages external feedback as a dynamic control variable to explicitly balance exploration and exploitation within the semantic space. |
| Outcome: | Experiments on Tau Bench and SearchQA show that the proposed framework outperforms standard RL baselines. |