Papers by Marcel Nawrath
On the Role of Summary Content Units in Text Summarization Evaluation (2024.naacl-short)
Copied to clipboard
Marcel Nawrath, Agnieszka Nowak, Tristan Ratz, Danilo Walenta, Juri Opitz, Leonardo Ribeiro, João Sedoc, Daniel Deutsch, Simon Mille, Yixin Liu, Sebastian Gehrmann, Lining Zhang, Saad Mahamood, Miruna Clinciu, Khyathi Chandu, Yufang Hou
| Challenge: | a human written summary content unit (SCU) is used to judge the quality of a summary . a pyramid evaluation method is based on SCUs that decompose a reference summary into concise sentences . |
| Approach: | They propose to use automated SCUs to evaluate the quality of a candidate summary . they propose to generate SCU approximations from AMR meaning representations and large language models . |
| Outcome: | The proposed method can be fully automated, but lacks the human effort to validate it. |