Papers by Lorenzo Proietti
Analyzing Homonymy Disambiguation Capabilities of Pretrained Language Models (2024.lrec-main)
Copied to clipboard
Lorenzo Proietti, Stefano Perrella, Simone Tedeschi, Giulia Vulpis, Leonardo Lavalle, Andrea Sanchietti, Andrea Ferrari, Roberto Navigli
| Challenge: | Word Sense Disambiguation (WSD) is a key task in Natural Language Processing (NLP) but current pretrained language models lack the granularity to perform disambiguation . |
| Approach: | They propose a large-scale resource that leverages homonymy relations to cluster WordNet senses and train Homonymy Disambiguation systems. |
| Outcome: | The proposed model can distinguish homonyms with up to 95% accuracy even without fine-tuning the underlying PLM. |
Beyond Correlation: Interpretable Evaluation of Machine Translation Metrics (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have shown that MT metrics return assessments as scalar scores that are difficult to interpret, posing a challenge to making informed design choices. |
| Approach: | They propose an interpretable evaluation framework that evaluates MT metrics in two scenarios that serve as proxies for filtering and translation re-ranking use cases. |
| Outcome: | The proposed framework offers clearer insights than correlation with human judgments. |
PEAR: Pairwise Evaluation for Automatic Relative Scoring in Machine Translation (2026.acl-long)
Copied to clipboard
| Challenge: | PEAR is a supervised quality estimation metric that reframes reference-free machine translation evaluation as a graded pairwise comparison. |
| Approach: | They propose to use a supervised quality estimation metric family to reframe machine translation evaluation as a graded pairwise comparison. |
| Outcome: | The proposed metric outperforms strictly matched single-candidate QE baselines on the WMT24 meta-evaluation benchmark. |
Estimating Machine Translation Difficulty (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Despite the high-quality outputs, it is difficult to distinguish between state-of-the-art models and identify areas for future improvement. |
| Approach: | They propose a new metric to evaluate difficulty estimators and use it to assess both baselines and novel approaches. |
| Outcome: | The proposed models outperform both heuristic-based methods and LLM-as-a-judge approaches, with sentinel-src achieving the best performance. |
Has Machine Translation Evaluation Achieved Human Parity? The Human Reference and the Limits of Progress (2025.acl-short)
Copied to clipboard
| Challenge: | In machine translation evaluation, metric performance is assessed based on agreement with human judgments. |
| Approach: | They incorporate human baselines into the MT meta-evaluation to gain a clearer understanding of metric performance and establish an upper bound. |
| Outcome: | The results suggest human parity, but there are several reasons to caution . |