On Context Utilization in Summarization with Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models excel in abstractive summarization tasks, delivering fluent and pertinent summaries. |
| Approach: | They conduct the first comprehensive study on context utilization and position bias in summarization. |
| Outcome: | The proposed benchmark compares two methods to alleviate position bias in summarization tasks. |
Similar Papers
PoSum-Bench: Benchmarking Position Bias in LLM-based Conversational Summarization (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models exhibit positional bias, a problem that can undermine the completeness of conversation summarizations. |
| Approach: | They propose a semantic similarity-based sentence-level metric to quantify positional bias in conversational summaries. |
| Outcome: | The proposed benchmark provides the first systematic evaluation of positional bias in conversational summarization across languages and contexts. |
On Positional Bias of Faithfulness for Long-form Summarization (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models exhibit positional bias in long-context settings, under-attending to information in the middle. |
| Approach: | They compile eight human-annotated long-form summarization datasets to evaluate faithfulness . they find that LLMs faithfully summarize beginning and end of documents but neglect middle content . |
| Outcome: | The proposed methods show that LLMs under-attend to information in the middle of inputs. |
CNNSum: Exploring Long-Context Summarization with Large Language Models in Chinese Novels (2025.findings-acl)
Copied to clipboard
| Challenge: | Currently, long-context summarization mainly relies on memory ability. |
| Approach: | They propose a multi-scale long-context summarization benchmark based on Chinese novels . they use human-driven annotations to analyze long-constituency models . |
| Outcome: | The proposed benchmark features human-driven annotations across four subsets with lengths ranging from 16k to 128k. |
What Matters to an LLM? Behavioral and Computational Evidences from Summarization (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly entrusted with the management of information. |
| Approach: | They combine behavioral and computational analyses to find out what LLMs prioritize . they generate length-controlled summaries and derive empirical importance distributions . |
| Outcome: | The proposed model converges on consistent importance patterns and clusters more by family than by size. |
Bias in News Summarization: Measures, Pitfalls and Corpora (2024.findings-acl)
Copied to clipboard
| Challenge: | Pretrained large language models can reproduce harmful social biases in constrained settings, such as summarization. |
| Approach: | They propose a method to generate input documents with carefully controlled demographic attributes and then apply it to a controlled setting. |
| Outcome: | The proposed method allows to generate input documents with carefully controlled demographic attributes while working with real-world input documents. |
Revisiting Zero-Shot Abstractive Summarization in the Era of Large Language Models from the Perspective of Position Bias (2024.naacl-short)
Copied to clipboard
| Challenge: | Position bias is a tendency of a model unfairly prioritizing information from certain parts of the input text over others, leading to undesirable behavior. |
| Approach: | They propose to measure position bias in large language models for zero-shot summarization tasks by measuring position bias. |
| Outcome: | The proposed model performance and position biases lead to new insights and discussion on zero-shot summarization tasks. |
Not Lost After All: How Cross-Encoder Attribution Challenges Position Bias Assumptions in LLM Summarization (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Position bias is a key limitation in automatic summarization. |
| Approach: | They propose a cross-encoder-based alignment method that processes summary-source sentence pairs . |
| Outcome: | The proposed method allows better identification of semantic correspondences even when summaries substantially rewrite the source. |
TriSum: Learning Summarization Ability from Large Language Models with Structured Rationale (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have advanced tasks like text summarization, but their size and computational demands limit their use in resource-constrained and privacy-centric settings. |
| Approach: | They propose a framework for distilling LLMs’ text summarization abilities into a compact, local model using a curriculum learning strategy that evolves from simple to complex tasks. |
| Outcome: | The proposed framework outperforms baseline models on CNN/DailyMail, XSum, and ClinicalTrial, and improves interpretability by providing insights into the summarization rationale. |
On Learning to Summarize with Large Language Models as References (2024.naacl-long)
Copied to clipboard
Yixin Liu, Kejian Shi, Katherine He, Longtian Ye, Alexander Fabbri, Pengfei Liu, Dragomir Radev, Arman Cohan
| Challenge: | Recent studies have found that summaries generated by large language models (LLMs) are favored by human annotators when compared to reference summary from widely used summarization datasets. |
| Approach: | They propose to use large language models (LLMs) as reference learning settings for smaller text summarization models to investigate whether their performance can be substantially improved. |
| Outcome: | The proposed model outperforms standard supervised fine-tuning and human evaluations while retaining human-level performance. |
Exploring Context Strategies in LLMs for Discourse-Aware Machine Translation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models excel at machine translation, but the impact of how LLMs utilize different forms of contextual information on discourse-level phenomena remains underexplored. |
| Approach: | They examine how different forms of context influence standard MT metrics and specific discourse phenomena such as formality, pronoun selection, and lexical cohesion. |
| Outcome: | Evaluating multiple LLMs across multiple domains and language pairs, the findings consistently show that context boosts translation and discourse-specific performance. |