Papers by Théo Gigant
Mitigating the Impact of Reference Quality on Evaluation of Summarization Systems with Reference-Free Metrics (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing metrics for summarization are reference-based and correlate poorly with relevance . fluency, faithfulness, coherence and relevance are all measures of human evaluation . |
| Approach: | They propose a reference-free metric that correlates well with human evaluated relevance . n-gram importance weighting is used to weight a summary's importance . |
| Outcome: | The proposed metric can be used along reference-based metrics to improve their robustness in low quality reference settings. |