Papers by Yannick Parmentier
An Error Analysis Framework for Shallow Surface Realization (2021.tacl-1)
Copied to clipboard
| Challenge: | BLEU and METEOR metrics fail to provide information on which linguistic factors impact performance of natural language generation models. |
| Approach: | They propose a framework for error analysis which permits identifying which features of the input affect the models’ results. |
| Outcome: | The proposed framework improves the performance of 174 system runs submitted to the Multilingual SR shared tasks. |
Semantic Evaluation of Multilingual Data-to-Text Generation via NLI Fine-Tuning: Precision, Recall and F1 scores (2025.findings-acl)
Copied to clipboard
| Challenge: | KG-to-Text models are prone to errors like Additions and Omissions, and few languages are taken into account since both train and test data are not readily available. |
| Approach: | They propose a multilingual evaluation framework that is reference-less . it allows estimating how much a KG-to-Text Model under- (omission) or over- (addition) generates. |
| Outcome: | The proposed evaluation framework outperforms prior reference-less metrics in correlation with human judgments and provides scores for precision and recall. |