SARA: Salience-Aware Reinforced Adaptive Decoding for Large Language Models in Abstractive Summarization (2025.acl-long)
Copied to clipboard
| Challenge: | Existing decoding strategies neglect the explicit use of salient contextual information and rely on static hyperparameters to fix the balance between contextual and prior knowledge. |
| Approach: | They propose a salience-aware reinforced adaptive decoding (SARA) which incorporates salient contextual information and allows the model to determine reliance on source document's context, salient context, and model's prior knowledge based on pointwise mutual information. |
| Outcome: | The proposed model improves the quality and faithfulness of summaries across LLMs without modifying their weights. |
Similar Papers
Dynamic Attention-Guided Context Decoding for Mitigating Context Faithfulness Hallucinations in Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods, such as a n-terminal coding, do not provide accurate data for large language models. |
| Approach: | They propose a lightweight framework that leverages attention distributions and uncertainty signals in a single-pass decoding. |
| Outcome: | Experiments on open-book QA datasets show that DAGCD improves faithfulness and robustness while preserving computational efficiency. |
CoCoA: Confidence- and Context-Aware Adaptive Decoding for Resolving Knowledge Conflicts in Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing contrastive decoding methods that handle conflict lack adaptability and can degrade performance in low conflict settings. |
| Approach: | They propose a token-level algorithm for principled conflict resolution and enhanced faithfulness that resolves conflict by utilizing confidence-aware measures and the generalized divergence between parametric and contextual distributions. |
| Outcome: | The proposed algorithm achieves 9.2 points on average in QA, summarization, and long-form question answering (LFQA) benchmarks and improves factuality by 2.5 points on the key benchmarks. |
Dynamic PMI-Guided Contrastive Decoding Reduces Hallucination in Large Language Models: A Unified Framework of Fine-Grained Input Transformations (2026.findings-acl)
Copied to clipboard
| Challenge: | Despite the remarkable generation capabilities of large language models, the issue of hallucination remains a critical challenge. |
| Approach: | They propose a contrastive decoding framework based on dynamic pointwise mutual information that disentangles spurious dependencies induced by context priors, lexical co-occurrences, and syntactic structures and prioritizes causal logic. |
| Outcome: | The proposed framework significantly improves the model’s factuality and reasoning robustness while maintaining high computational efficiency. |
Mitigating Hallucination in Abstractive Summarization with Domain-Conditional Mutual Information (2024.findings-naacl)
Copied to clipboard
| Challenge: | Prior studies have attempted to enhance faithfulness of abstractive summarization, yet hallucination remains a persistent challenge. |
| Approach: | They propose a decoding strategy that adjusts the generation probability of each token by comparing it with the token’s marginal probability within the domain of the source text. |
| Outcome: | The proposed method significantly improves faithfulness and source relevance on the XSUM dataset. |
Mutual Information Alleviates Hallucinations in Abstractive Summarization (2022.emnlp-main)
Copied to clipboard
| Challenge: | Abstractive summarization models exhibit the tendency to hallucinate, i.e., output content not supported by the source document. |
| Approach: | They propose a decoding strategy that optimizes for pointwise mutual information of source and target tokens when models exhibit uncertainty. |
| Outcome: | The proposed method decreases the probability of hallucinated tokens while maintaining the Rouge and BERT-S scores of top-performing decoding strategies. |
Improving Abstractive Document Summarization with Salient Information Modeling (P19-1)
Copied to clipboard
| Challenge: | Abstractive document summarization is a task of natural language generation which generates fluent summaries with salient information automatically. |
| Approach: | They propose to incorporate a Gaussian focal bias on attention scores into an encoder to enhance the perception of local context and to distinguish salient information precisely. |
| Outcome: | The proposed framework outperforms state-of-the-art models on the CNN/Daily Mail benchmark and is based on a focus-attention mechanism and two new extensions. |
AdaFuse: Adaptive Ensemble Decoding for Large Language Models (2026.acl-long)
Copied to clipboard
Chengming Cui, Tianxin Wei, Ziyi Chen, Ruizhong Qiu, Zhichen Zeng, Zhining Liu, Xuying Ning, Duo Zhou, Jingrui He
| Challenge: | Existing ensemble approaches to large language models lack flexibility for mid-generation adaptation. |
| Approach: | They propose an adaptive ensemble decoding framework that dynamically selects semantically appropriate fusion units during generation. |
| Outcome: | The proposed framework outperforms existing ensemble frameworks on open-domain QA, arithmetic reasoning, and machine translation tasks. |
Trusting Your Evidence: Hallucinate Less with Context-aware Decoding (2024.naacl-short)
Copied to clipboard
| Challenge: | Language models (LMs) often struggle to pay enough attention to the input context, and generate texts that are unfaithful or contain hallucinations. |
| Approach: | They propose a context-aware decoding technique that amplifies the difference between the output probabilities when a model is used with and without context. |
| Outcome: | The proposed model significantly improves faithfulness of different LM families including OPT, GPT, LLaMA, and FLAN-T5 for summarization tasks. |
Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting (P18-1)
Copied to clipboard
| Challenge: | Empirically, we achieve the new state-of-the-art on all metrics (including human evaluation) on the CNN/Daily Mail dataset, as well as significantly higher abstractiveness scores. |
| Approach: | They propose a sentence-level policy gradient method that bridges computation between two neural networks in a hierarchical way while maintaining language fluency. |
| Outcome: | The proposed model achieves state-of-the-art on all metrics and higher abstractiveness scores on the CNN/Daily Mail dataset and faster training convergence than previous models. |
Topic-Guided Reinforcement Learning with LLMs for Enhancing Multi-Document Summarization (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown impressive results in single-document summarization, but their performance on MDS still leaves room for improvement. |
| Approach: | They propose a topic-guided reinforcement learning approach to improve content selection in MDS . explicit prompting models with topic labels enhances the informativeness, they show . |
| Outcome: | The proposed method outperforms baselines on multi-News and multi-XScience datasets. |