Papers by Stefano Perrella
Analyzing Homonymy Disambiguation Capabilities of Pretrained Language Models (2024.lrec-main)
Copied to clipboard
Lorenzo Proietti, Stefano Perrella, Simone Tedeschi, Giulia Vulpis, Leonardo Lavalle, Andrea Sanchietti, Andrea Ferrari, Roberto Navigli
| Challenge: | Word Sense Disambiguation (WSD) is a key task in Natural Language Processing (NLP) but current pretrained language models lack the granularity to perform disambiguation . |
| Approach: | They propose a large-scale resource that leverages homonymy relations to cluster WordNet senses and train Homonymy Disambiguation systems. |
| Outcome: | The proposed model can distinguish homonyms with up to 95% accuracy even without fine-tuning the underlying PLM. |
Beyond Correlation: Interpretable Evaluation of Machine Translation Metrics (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have shown that MT metrics return assessments as scalar scores that are difficult to interpret, posing a challenge to making informed design choices. |
| Approach: | They propose an interpretable evaluation framework that evaluates MT metrics in two scenarios that serve as proxies for filtering and translation re-ranking use cases. |
| Outcome: | The proposed framework offers clearer insights than correlation with human judgments. |
ConLoan: A Contrastive Multilingual Dataset for Evaluating Loanwords (2025.acl-long)
Copied to clipboard
Sina Ahmadi, Micha David Hess, Elena Álvarez-Mellado, Alessia Battisti, Cui Ding, Anne Göhring, Yingqiang Gao, Zifan Jiang, Andrianos Michail, Peshmerge Morad, Joel Niklaus, Maria Christina Panagiotopoulou, Stefano Perrella, Juri Opitz, Anastassia Shaitarova, Rico Sennrich
| Challenge: | Lexical borrowing is a ubiquitous linguistic phenomenon influenced by geopolitical, societal, and technological factors. |
| Approach: | They propose a novel contrastive dataset comprising sentences with and without loanwords across 10 languages to examine how machine translation and language models process loanword . |
| Outcome: | The proposed dataset shows that state-of-the-art models prefer loanwords over native terms and exhibit varying performance across languages. |
Estimating Machine Translation Difficulty (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Despite the high-quality outputs, it is difficult to distinguish between state-of-the-art models and identify areas for future improvement. |
| Approach: | They propose a new metric to evaluate difficulty estimators and use it to assess both baselines and novel approaches. |
| Outcome: | The proposed models outperform both heuristic-based methods and LLM-as-a-judge approaches, with sentinel-src achieving the best performance. |
Has Machine Translation Evaluation Achieved Human Parity? The Human Reference and the Limits of Progress (2025.acl-short)
Copied to clipboard
| Challenge: | In machine translation evaluation, metric performance is assessed based on agreement with human judgments. |
| Approach: | They incorporate human baselines into the MT meta-evaluation to gain a clearer understanding of metric performance and establish an upper bound. |
| Outcome: | The results suggest human parity, but there are several reasons to caution . |