Papers by Junsoo Park
HaRiM+: Evaluating Summary Quality with Hallucination Risk (2022.aacl-main)
Copied to clipboard
| Challenge: | Existing summarization models are limited in measuring the factual inconsistency of generated summaries. |
| Approach: | They propose a decoder overconfidence-regularizing objective as a hallucination risk measurement to better estimate the quality of generated summaries. |
| Outcome: | The proposed metric is reference-free and requires no training or modules . it records state-of-the-art correlation to human judgment on three sets of summary-quality annotations. |
OffsetBias: Leveraging Debiased Data for Tuning Evaluators (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have identified that judge models are vulnerable to certain biases, such as favoring longer answers regardless of content. |
| Approach: | They propose to use open-source LLMs to fine-tune judge models and propose a meta-evaluation collection of test cases for each bias type. |
| Outcome: | The proposed meta-evaluation dataset shows that fine-tuning improves the robustness of judge models against biases and improves performance across most evaluation scenarios. |