GMSA: Enhancing Context Compression via Group Merging and Layer Semantic Alignment (2026.acl-long)
Copied to clipboard
Jiwei Tang, Zhicheng Zhang, Shunlong Wu, Jingheng Ye, Lichen Bai, Zitai Wang, Tingwei Lu, Lin Hai, Yiming Zhao, Hai-Tao Zheng, Hong-Gee Kim
| Challenge: | Large Language Models (LLMs) have achieved remarkable performance across NLP tasks . however, in long-context scenarios, they face high computational cost and information redundancy. |
| Approach: | They propose an encoder-decoder context compression framework that generates a compact sequence of soft tokens for downstream tasks. |
| Outcome: | Experiments show that GMSA outperforms baselines on multiple long-context question answering and summarization benchmarks while maintaining low end-to-end latency. |
Similar Papers
A Silver Bullet or a Compromise for Full Attention? A Comprehensive Study of Gist Token-based Context Compression (2025.acl-long)
Copied to clipboard
| Challenge: | gist-based context compression methods can achieve only slight performance loss on tasks like retrieval-augmented generation and long-document QA, but it faces challenges in tasks like synthetic recall. |
| Approach: | They propose two strategies to improve gist-based context compression in large language models. |
| Outcome: | The proposed methods can achieve only slight performance loss on retrieval-augmented generation and long-document QA tasks, but they face challenges in tasks like synthetic recall. |
DAST: Context-Aware Compression in LLMs via Dynamic Allocation of Soft Tokens (2025.findings-acl)
Copied to clipboard
Shaoshen Chen, Yangning Li, Zishan Xu, Yongqin Zeng, Shunlong Wu, Xinshuo Hu, Zifei Shan, Xin Su, Jiwei Tang, Yinghui Li, Hai-Tao Zheng
| Challenge: | Existing semantic vector-based compression methods do not account for the intrinsic information density variations between context chunks, instead allocating soft tokens uniformly across context chunk. |
| Approach: | They propose a method that leverages the LLM's intrinsic understanding of contextual relevance to guide compression. |
| Outcome: | The proposed method surpasses state-of-the-art methods on long context tasks. |
Bridging the Memorization-Utilization Gap: Near-Lossless Context Compression via Reinforcement Learning (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in context compression have failed to effectively utilize compressed representations for downstream tasks. |
| Approach: | They propose a holistic training paradigm that uses outcome-based RL to enable implicit expansion. |
| Outcome: | The proposed model outperforms previous models on NIAH, LongBench and multi-hop reasoning. |
Extending Context Window of Large Language Models via Semantic Compression (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing models rely on a quadratic computation to generate long texts . current models impose limitations on the length of text inputs . |
| Approach: | They propose a semantic compression method that extends the context window of large language models . the method reduces the semantic redundancy of long inputs before passing them to the LLMs . |
| Outcome: | The proposed method extends the context window of large language models across tasks . it exhibits consistent fluency in text generation while reducing associated computational overhead. |
Position IDs Matter: An Enhanced Position Layout for Efficient Context Compression in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to compress context information ignore holistic contextual dependencies. |
| Approach: | They propose a method that adjusts position encodings to minimize the distance between context tokens and special tokens. |
| Outcome: | Enhanced Position Layout (EPL) improves compression of context information in large language models. |
Pretraining Context Compressor for Large Language Models with Embedding-Based Memory (2025.acl-long)
Copied to clipboard
| Challenge: | Efficient processing of long contexts in large language models is essential for real-world applications such as retrieval-augmented generation and in-context learning. |
| Approach: | They propose a decoupled compressor-LLM framework that preserves contextual information within condensed embedding representations. |
| Outcome: | The proposed framework outperforms baseline models in three domains and across eight datasets while adapting to different downstream LLMs. |
LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly lengthy and require longer prompts . this paper presents a coarse-to-fine prompt compression method to reduce cost and increase performance. |
| Approach: | They propose a coarse-to-fine prompt compression method that maintains semantic integrity under high compression ratios and a token-level iterative compression algorithm to better model the interdependence between compressed contents. |
| Outcome: | The proposed method yields state-of-the-art performance and allows for up to 20x compression with little performance loss over four datasets from different scenarios. |
Hierarchical Token Prepending: Enhancing Information Flow in Decoder-based LLM Embeddings (2026.acl-long)
Copied to clipboard
Xueying Ding, Xingyue Huang, Mingxuan Ju, Liam Collins, Yozen Liu, Leman Akoglu, Neil Shah, Tong Zhao
| Challenge: | Large language models produce powerful text embeddings, but their causal attention mechanism restricts the flow of information from later to earlier tokens, harming performance. |
| Approach: | They propose a method that prepending a single summary token to reduce attention-level compression by partitioning the input into blocks and prepending blocks to subsequent blocks. |
| Outcome: | The proposed method achieves consistent performance gains across 11 retrieval datasets and 30 general embedding benchmarks. |
Context Compression for Auto-regressive Transformers with Sentinel Tokens (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing Transformer-based LLMs have limited performance due to complexity of attention module . key-value cache is the major memory footprint and inference latency problem . |
| Approach: | They propose a plug-and-play approach that incrementally compresses token activation into compact ones . they also profile the benefit of context compression on improving the system throughout . |
| Outcome: | The proposed approach reduces memory footprint and inference latency by compressing tokens into compact ones. |
Adapting Language Models to Compress Contexts (2023.emnlp-main)
Copied to clipboard
| Challenge: | Transformer-based language models have a finite context window and expensive computational cost of processing long text documents. |
| Approach: | They propose to adapt pre-trained LMs into AutoCompressors to compress text into summary vectors . authors propose to use summary vector to speed up inference over long contexts based on a finite context window . |
| Outcome: | The proposed model can compress long contexts into summary vectors, which are accessible as soft prompts. |