Model-based Preference Optimization in Abstractive Summarization without Human Feedback (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) can generate fluent summaries but often introduce inaccuracies by hallucinating content not found in the source document. |
| Approach: | They propose a method to fine-tune Large Language Models for improved summarization abilities without any human feedback. |
| Outcome: | The proposed method significantly improves the quality of generated summaries without any human feedback. |
Similar Papers
Make The Most of Prior Data: A Solution for Interactive Text Summarization with Preference Feedback (2022.findings-naacl)
Copied to clipboard
Duy-Hung Nguyen, Nguyen Viet Dung Nghiem, Bao-Sinh Nguyen, Dung Tien Tien Le, Shahab Sabahi, Minh-Tien Nguyen, Hung Le
| Challenge: | a framework to train summarization models with preference feedback is proposed . human-in-the-loop (HITL) allows humans to actively participate in supervising AI systems . |
| Approach: | They propose a framework to train summarization models with preference feedback interactively. |
| Outcome: | The proposed framework improves ROUGE scores and sample-efficiency in active, few-shot and online settings. |
Can Language Models Capture Human Writing Preferences for Domain-Specific Text Summarization? (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent studies employ large language models as auxiliary tools for humancentered NLP. |
| Approach: | They construct a model to capture human writing preferences by fine-tuning pre-trained models with data and designing prompts to optimize the output of large language models. |
| Outcome: | The proposed model captures human writing preferences through the dimensions of length, content depth, tone & style, and summary format. |
Learning to Summarize from LLM-generated Feedback (2025.naacl-long)
Copied to clipboard
| Challenge: | Developing effective text summarizers remains a challenge due to issues like unfaithful statements, key information omissions, and verbosity. |
| Approach: | They propose a large-scale dataset containing multi-dimensional feedback on LLM-generated summaries of varying quality across diverse domains to align them with human preferences for faithfulness, completeness, and conciseness. |
| Outcome: | The proposed model outperforms the 10x larger Llama3-70b-instruct in generating human-preferred summaries. |
Abstractive Document Summarization with Summary-length Prediction (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing abstractive summarization models do not consider summarizing-specific information such as the target summary length. |
| Approach: | They propose a method for enabling a model to understand summarization-specific information by predicting the summary length in the encoder and generating a summary of the predicted length in fine-tuning. |
| Outcome: | The proposed method improves ROUGE scores on the WikiHow, NYT, and CNN/DM datasets. |
Towards Enhancing Coherence in Extractive Summarization: Dataset and Experiments with LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for extractive summarization lack coherence, despite improvements . a human-annotated dataset is used to improve coherency of extractive summary . |
| Approach: | They propose to use human-annotated datasets to create coherent extractive summaries . they use supervised fine-tuning and natural language user feedback to enhance coherence . |
| Outcome: | The proposed dataset shows that LLMs can produce coherent summaries with human feedback. |
PrefScore: Pairwise Preference Learning for Reference-free Summarization Quality Assessment (2022.coling-1)
Copied to clipboard
| Challenge: | Existing studies on summarization evaluation without a human-written reference summary have shown high correlations with human ratings. |
| Approach: | They propose to judge summary quality by learning preference rank from corrupted summaries . they use Bradley-Terry power ranking model to learn preference rank . |
| Outcome: | Experiments on several datasets show that the proposed model can produce scores highly correlated with human ratings. |
A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents (N18-2)
Copied to clipboard
Arman Cohan, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Seokhwan Kim, Walter Chang, Nazli Goharian
| Challenge: | Existing abstractive summarization models focus on summarizing sentences and short documents. |
| Approach: | They propose a hierarchical encoder that models the discourse structure of a document, and an attentive discourse-aware decoder to generate the summary. |
| Outcome: | The proposed model significantly outperforms state-of-the-art models on two large-scale datasets of scientific papers. |
Improving Factual Consistency of News Summarization by Contrastive Preference Optimization (2024.findings-emnlp)
Copied to clipboard
Huawen Feng, Yan Fan, Xiong Liu, Ting-En Lin, Zekun Yao, Yuchuan Wu, Fei Huang, Yongbin Li, Qianli Ma
| Challenge: | Recent advances in news summarization have created problems with “hallucinations” that are factually inconsistent with the source text. |
| Approach: | They propose to disentangle LLMs’ propensities to generate faithful and fake content by adopting a probing-based specific training method to improve their capacity of distinguishing two types of propensity. |
| Outcome: | The proposed method disentangles LLMs’ propensities to generate faithful and fake content and improves their ability to distinguish between two types of propensity. |
Improving Fairness of Large Language Models in Multi-document Summarization (2025.acl-short)
Copied to clipboard
| Challenge: | Recent studies focus on summary-level fairness, while corpus-level focuses on corpus of summaries. |
| Approach: | They propose a preference tuning method that focuses on both summary-level and corpus-level fairness in MDS. |
| Outcome: | The proposed method outperforms baselines while maintaining critical qualities of summaries. |
An Empirical Study of Many-to-Many Summarization with Large Language Models (2025.acl-long)
Copied to clipboard
Jiaan Wang, Fandong Meng, Zengkui Sun, Yunlong Liang, Yuxuan Cao, Jiarong Xu, Haoxiang Shi, Jie Zhou
| Challenge: | Recent studies have shown that large language models (LLMs) have strong multilingual abilities, giving them the potential to perform M2MS in real applications. |
| Approach: | They propose to use many-to-many summarization (M2MS) to generate a brief summary in any language given a document also in any other language. |
| Outcome: | The proposed model outperforms zero-shot LLMs in terms of automatic evaluations. |