Papers by Nuno Guerreiro
Enhanced Hallucination Detection in Neural Machine Translation through Simple Detector Aggregation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) systems suffer from various pathologies, including the generation of translations that are detached from the source content, typically known as hallucinations. |
| Approach: | They propose to combine detectors and introduce a method for aggregating detectors to detect hallucinations. |
| Outcome: | The proposed method provides a promising step towards evermore reliable machine translation systems. |
xTower: A Multilingual LLM for Explaining and Correcting Translation Errors (2024.findings-emnlp)
Copied to clipboard
Marcos Treviso, Nuno Guerreiro, Sweta Agrawal, Ricardo Rei, José Pombal, Tania Vaz, Helena Wu, Beatriz Silva, Daan Stigt, Andre Martins
| Challenge: | Neural machine translation systems produce translations with errors and anomalies . understanding these errors can help improve the translation quality and user experience . |
| Approach: | They propose an open large language model (LLM) built on top of TowerBase to provide free-text explanations for translation errors in order to guide the generation of a corrected translation. |
| Outcome: | The proposed model improves translation quality and user experience by allowing translators to provide free-text explanations for errors and anomalies. |
Steering Large Language Models for Machine Translation with Finetuning and In-Context Learning (2023.findings-emnlp)
Copied to clipboard
Duarte Alves, Nuno Guerreiro, João Alves, José Pombal, Ricardo Rei, José de Souza, Pierre Colombo, Andre Martins
| Challenge: | Large language models (LLMs) are a promising avenue for machine translation (MT) however, their effectiveness depends on the choice of few-shot examples and they often require extra post-processing due to overgeneration. |
| Approach: | They propose a method that incorporates few-shot examples during finetuning to improve performance on MT tasks. |
| Outcome: | The proposed method outperforms few-shot prompting while eliminating the need for in-context examples. |
Analyzing Context Contributions in LLM-based Machine Translation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have achieved state-of-the-art performance in machine translation . however, the mechanisms by which LLMs use different parts of the input context remain unexplored . |
| Approach: | They propose to analyze how large language models use different parts of the input context . they highlight several key findings: the source part of few-shot examples contributes more than its corresponding targets . |
| Outcome: | The proposed model can leverage in-context learning to perform translation tasks without training . the proposed model is able to perform tasks without being explicitly trained for them . |
Modeling User Preferences with Automatic Metrics: Creating a High-Quality Preference Dataset for Machine Translation (2024.emnlp-main)
Copied to clipboard
Sweta Agrawal, José De Souza, Ricardo Rei, António Farinhas, Gonçalo Faria, Patrick Fernandes, Nuno Guerreiro, Andre Martins
| Challenge: | Existing algorithms for machine translation do not match human preferences, but they can be expensive to obtain and curate at a large scale. |
| Approach: | They propose an approach that leverages the best of both worlds by collecting sentence-level quality assessments from professional linguists on translations generated by multiple high-quality MT systems. |
| Outcome: | The proposed approach improves translation quality on WMT23 and FLORES benchmarks. |