Papers with XSUM
ELMER: A Non-Autoregressive Pre-trained Language Model for Efficient and Effective Text Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for text generation use auto-regressive (AR) methods, but inefficient inference is a problem. |
| Approach: | They propose an efficient and effective PLM to explicitly model the token dependency during NAR text generation. |
| Outcome: | The proposed model outperforms existing models on three text generation tasks while achieving 10 times faster inference speedup. |
Improving Factuality of Abstractive Summarization without Sacrificing Summary Quality (2023.acl-short)
Copied to clipboard
| Challenge: | Recent studies have shown that most abstractive summarization models are unfaithful and suffer from a wide range of hallucination. |
| Approach: | They propose a candidate summary generation and ranking technique to improve summary factuality without sacrificing quality. |
| Outcome: | The proposed method shows that the model trained using the proposed method improves on factuality and similarity-based metrics without conflicting with the model. |
Jointly Learning Guidance Induction and Faithful Summary Generation via Conditional Variational Autoencoders (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for abstractive summarization generate factual consistency summaries with a high level of accuracy and coherence. |
| Approach: | They propose a framework that induces the guidance information and generates summary equipment with the guidance synchronously. |
| Outcome: | The proposed framework generates fluent summaries with no constraint on the words and phrases, and is more faithful than the existing state-of-the-art approaches. |
Asking and Answering Questions to Evaluate the Factual Consistency of Summaries (2020.acl-main)
Copied to clipboard
| Challenge: | Existing automatic evaluation metrics for summarization are insensitive to factual inconsistencies. |
| Approach: | They propose an automatic evaluation protocol that detects factual inconsistencies in a model-generated summary. |
| Outcome: | QAGS has higher correlations with human judgments of factual consistency than other evaluation metrics. |
Segmented Recurrent Transformer: An Efficient Sequence-to-Sequence Model (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Transformers have shown dominant performance across a range of domains including language and vision, but their computational cost grows quadratically with the sequence length, making their usage prohibitive for resource-constrained applications. |
| Approach: | They propose a segmented recurrent transformer that combines segmente recursion with recursive attention to reduce the computational cost. |
| Outcome: | The proposed model achieves higher ROUGE1 scores and lower computational complexity than current approaches. |
CaPE: Contrastive Parameter Ensembling for Reducing Hallucination in Abstractive Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work suggests that the degree of hallucination depends on factual errors in training data. |
| Approach: | They propose a method to use training data to reduce hallucination by ensembling parameter variations in training data. |
| Outcome: | The proposed method improves on XSUM and CNN/DM datasets on human evaluations and factual metrics. |
Improving Consistency for Text Summarization with Energy Functions (2023.findings-emnlp)
Copied to clipboard
Qi Zeng, Qingyu Yin, Zheng Li, Yifan Gao, Sreyashi Nag, Zhengyang Wang, Bing Yin, Heng Ji, Chao Zhang
| Challenge: | Current abstractive summarization models generate inconsistent content due to the inherently noisy dataset and the discrepancy between maximum likelihood estimation based training objectives and consistency measurements. |
| Approach: | They propose a new consistency taxonomy that categorizes inconsistent content into faithfulness, factuality, and self-supportiveness. |
| Outcome: | Experiments on XSUM and CNN/DM datasets show that EnergySum mitigates the trade-off between accuracy and consistency. |
Exploration-Driven Reinforcement Learning for Expert Routing Improvement in Mixture-of-Experts Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | MoE-based LLMs are not explicitly supervised to select suitable experts. |
| Approach: | They propose Exploration-Driven Reinforcement Learning (ERL) which explicitly optimizes the router by exploration of alternative routing paths. |
| Outcome: | The proposed method improves summarization (SAMSum, XSUM, question answering, and language modeling), and raises routing quality, delivering 8.9 higher MRR than baselines over 100 perturbed routing paths. |
FractalLLM: Lossless Self-Speculative Decoding with Layer Embedded Self-Compression (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Autoregressive decoding requires a full forward pass for each generated token, increasing inference latency. |
| Approach: | They propose a lossless self-speculative decoding method that embeds a compressed model within selected decoder layers of the original model. |
| Outcome: | The proposed method achieves substantial speed-ups (up to 2.47) over standard autoregressive decoding. |