Papers by Patrick Fernandes
When Does Translation Require Context? A Data-driven, Multilingual Exploration (2023.acl-long)
Copied to clipboard
| Challenge: | Recent studies in context-aware MT attempt to target a small set of discourse phenomena during evaluation, however not in a fully systematic way. |
| Approach: | They develop a multilingual discourse-aware benchmark to evaluate model performance on discourse phenomena in a given dataset. |
| Outcome: | The proposed model improves on previously studied phenomena while uncovering others which were not addressed. |
Not-Just-Scaling Laws: Towards a Better Understanding of the Downstream Impact of Language Model Design Decisions (2025.emnlp-main)
Copied to clipboard
Emmy Liu, Amanda Bertsch, Lintang Sutawika, Lindia Tjuatja, Patrick Fernandes, Lara Marinov, Michael Chen, Shreya Singhal, Carolin Lawrence, Aditi Raghunathan, Kiril Gashteovski, Graham Neubig
| Challenge: | Language model performance is largely dependent on pretraining decisions, but scaling laws based on only these two aspects do not always explain downstream task performance. |
| Approach: | They meta-analyze 92 open-source pretrained models to quantify their impact on performance. |
| Outcome: | The framework lays a foundation for more systematic investigation of how model development choices shape final capabilities. |
A Context-aware Framework for Translation-mediated Conversations (2026.tacl-1)
Copied to clipboard
| Challenge: | Existing systems that bridge language barriers can introduce errors leading to misunderstandings and conversation breakdown. |
| Approach: | They propose a framework to integrate contextual information into automatic translation systems . they validate the framework on customer chat and user-assistant interaction . |
| Outcome: | The proposed framework consistently produces better translations than state-of-the-art systems on two task-oriented domains. |
Fine-Grained Reward Optimization for Machine Translation using Error Severity Mappings (2026.tacl-1)
Copied to clipboard
Miguel Moura Ramos, Tomás Almeida, Daniel Vareta, Filipe Azevedo, Sweta Agrawal, Patrick Fernandes, André F. T. Martins
| Challenge: | Reinforcement learning (RL) is an effective and robust method for training neural machine translation systems. |
| Approach: | They propose a method that leverages fine-grained, token-level quality assessments . they use a state-of-the-art quality estimation system as their token- level reward model . |
| Outcome: | The proposed approach leverages fine-grained, token-level quality assessments along with error severity levels to improve translation quality. |
ESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkit (2023.acl-demo)
Copied to clipboard
Brian Yan, Jiatong Shi, Yun Tang, Hirofumi Inaguma, Yifan Peng, Siddharth Dalmia, Peter Polák, Patrick Fernandes, Dan Berrebbi, Tomoki Hayashi, Xiaohui Zhang, Zhaoheng Ni, Moto Hira, Soumi Maiti, Juan Pino, Shinji Watanabe
| Challenge: | ESPnet-ST-v2 is a revamp of the open-source spoken language translation toolkit . it supports offline speech-to-text translation (ST), simultaneous speech- to-text (SST), and offline speech to-speech (S2ST) |
| Approach: | They propose to revamp the open-source ESPnet-ST toolkit to support offline speech-to-text translation, simultaneous speech- to-text and offline speech to-speech translation. |
| Outcome: | The updated version of ESPnet-ST supports offline speech-to-text translation (ST), simultaneous speech- to-text (SST), and offline speech to-speech translation (S2ST). |
Multi-Dimensional Evaluation of Text Summarization with In-Context Learning (2023.findings-acl)
Copied to clipboard
Sameer Jain, Vaishakh Keshava, Swarnashree Mysore Sathyendra, Patrick Fernandes, Pengfei Liu, Graham Neubig, Chunting Zhou
| Challenge: | In-context learning-based evaluators are competitive with learned evaluation frameworks for text summarization tasks. |
| Approach: | They propose to use large language models as multi-dimensional evaluators using in-context learning to evaluate text summarization tasks. |
| Outcome: | The proposed frameworks are competitive with existing frameworks on relevance and factual consistency, the authors show . |
Do Context-Aware Translation Models Pay the Right Attention? (2021.acl-long)
Copied to clipboard
| Challenge: | Context-aware machine translation models fail to leverage contextual information to resolve ambiguous words and pronouns. |
| Approach: | They propose a new dataset that includes supporting context words for 14K translations that professional translators found useful for pronoun disambiguation. |
| Outcome: | The proposed model can automatically disambiguate pronouns and polysemous words when they are not in the same context. |
A Multi-dimensional Evaluation of Tokenizer-free Multilingual Pretrained Models (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent work on tokenizer-free models shows promising results in cross-lingual transfer . previous work focused on reporting accuracy on a limited set of tasks and data settings . |
| Approach: | They compare tokenizer-free and subword-based models using various dimensions . they find subword models are still the most practical choice in many settings . |
| Outcome: | The proposed model improves cross-lingual transfer and reduces engineering overhead. |
Quality-Aware Decoding for Neural Machine Translation (2022.naacl-main)
Copied to clipboard
Patrick Fernandes, António Farinhas, Ricardo Rei, José G. C. de Souza, Perez Ogayo, Graham Neubig, Andre Martins
| Challenge: | Despite advances in machine translation quality estimation and evaluation, decoding is mostly oblivious to this. |
| Approach: | They propose to use a decoding framework that is quality-aware for neural machine translation . they compare various methods like N-best reranking and minimum Bayes risk decoding . |
| Outcome: | The proposed quality-aware decoding outperforms MAP-based decoding on four datasets and two model classes. |
Measuring and Increasing Context Usage in Context-Aware Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Recent work in neural machine translation has demonstrated the necessity and feasibility of using inter-sentential context, but it is often not clear how much they actually utilize it at translation time. |
| Approach: | They propose a conditional cross-mutual information metric to quantify usage of context by model architectures that can use it at translation time. |
| Outcome: | The proposed method increases context usage and improves translation quality according to BLEU and COMET metrics. |
Epsilon Sampling Rocks: Investigating Sampling Strategies for Minimum Bayes Risk Decoding for Machine Translation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in machine translation (MT) have shown that minimum bayes risk decoding can be a powerful alternative to beam search. |
| Approach: | They propose to use epsilon-sampling to prune away all tokens with a smaller probability mass. |
| Outcome: | The proposed method outperforms beam search decoding and other methods in four languages. |
Modeling User Preferences with Automatic Metrics: Creating a High-Quality Preference Dataset for Machine Translation (2024.emnlp-main)
Copied to clipboard
Sweta Agrawal, José De Souza, Ricardo Rei, António Farinhas, Gonçalo Faria, Patrick Fernandes, Nuno Guerreiro, Andre Martins
| Challenge: | Existing algorithms for machine translation do not match human preferences, but they can be expensive to obtain and curate at a large scale. |
| Approach: | They propose an approach that leverages the best of both worlds by collecting sentence-level quality assessments from professional linguists on translations generated by multiple high-quality MT systems. |
| Outcome: | The proposed approach improves translation quality on WMT23 and FLORES benchmarks. |