An Information Bottleneck Perspective for Effective Noise Filtering on Retrieval-Augmented Generation (2024.acl-long)
Copied to clipboard
Kun Zhu, Xiaocheng Feng, Xiyuan Du, Yuxuan Gu, Weijiang Yu, Haotian Wang, Qianglong Chen, Zheng Chu, Jingchang Chen, Bing Qin
| Challenge: | Experimental results show that retrieval-augmented generation improves accuracy and relevance of large language models. |
| Approach: | They propose to introduce the information bottleneck theory into retrieval-augmented generation by maximizing mutual information between compression and ground output while minimizing mutual information . |
| Outcome: | The proposed approach improves accuracy and correctness of answer generation and conciseness with 2.5% compression rate. |
Similar Papers
Familiarity-Aware Evidence Compression for Retrieval-Augmented Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Retrieval-augmented generation (RAG) improves large language models by incorporating non-parametric knowledge through evidence retrieved from external sources. |
| Approach: | They propose a training-free evidence compression technique that makes retrieved evidence more familiar to the target model while seamlessly integrating parametric knowledge from the model. |
| Outcome: | The proposed technique outperforms the most recent evidence compression baselines across open-domain QA datasets while achieving high compression rates. |
Searching for Best Practices in Retrieval-Augmented Generation (2024.emnlp-main)
Copied to clipboard
Xiaohua Wang, Zhenghua Wang, Xuan Gao, Feiran Zhang, Yixin Wu, Zhibo Xu, Tianyuan Shi, Zhengyuan Wang, Shizheng Li, Qi Qian, Ruicheng Yin, Changze Lv, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Retrieval-augmented generation (RAG) techniques have proven to be effective in integrating up-to-date information, mitigating hallucinations, and enhancing response quality, especially in specialized domains. |
| Approach: | They propose several strategies for deploying RAG that balance performance and efficiency. |
| Outcome: | The proposed approaches can significantly enhance question-answering capabilities and accelerate the generation of multimodal content using a “retrieval as generation” strategy. |
Exploring Knowledge Filtering for Retrieval-Augmented Discriminative Tasks (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have focused on generative tasks, while its potential in discriminative tasks remains largely unexplored. |
| Approach: | They propose a framework that incorporates knowledge filtering and prediction fusion mechanisms to improve model performance. |
| Outcome: | The proposed framework improves model performance on discriminative tasks by filtering out harmful knowledge and integrating it into the input context. |
Towards Knowledge Checking in Retrieval-augmented Generation: A Representation Perspective (2025.naacl-long)
Copied to clipboard
Shenglai Zeng, Jiankun Zhang, Bingheng Li, Yuping Lin, Tianqi Zheng, Dante Everaert, Hanqing Lu, Hui Liu, Hui Liu, Yue Xing, Monica Xiao Cheng, Jiliang Tang
| Challenge: | Existing studies have shown that LLMs struggle to identify the boundaries of their own knowledge and tend to prioritize external information over internal knowledge learned during pre-training. |
| Approach: | They conduct a comprehensive analysis of LLM representation behaviors and demonstrate the significance of using representations in knowledge checking. |
| Outcome: | The proposed classifiers improve performance even when dealing with noisy knowledge databases. |
CompAct: Compressing Retrieved Documents Actively for Question Answering (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to condense extensive documents with no loss of information are difficult to implement in real-world scenarios. |
| Approach: | They propose a framework that employs an active strategy to condense extensive documents without losing key information. |
| Outcome: | The proposed framework improves performance and compression rate on multi-hop question-answering benchmarks. |
Fine-grained Knowledge Enhancement for Retrieval-Augmented Generation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies rely on semantic similarity to retrieve knowledge but ignore fine-grained information within documents. |
| Approach: | They propose a fine-grained knowledge enhancement method to fill knowledge gaps with retrieved external information by a Chain-of-Thought prompting procedure and a decoding enhancement strategy to constrain the document-based decoding process. |
| Outcome: | The proposed method can be applied in a plug-and-play manner to enhance its performance with no additional modules or training process. |
Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work has proposed to improve relevance modeling by having large language models actively involved in retrieval, i.e., to guide retrieval with generation. |
| Approach: | They propose to have large language models actively involved in retrieval to guide retrieval with generation. |
| Outcome: | The proposed method synergizes retrieval and generation in an iterative manner, and can generate better results in subsequent iterations. |
Optimizing Retrieval-augmented Reader Models via Token Elimination (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for ODQA use a retrieval-augmented language model . a generative model can cause a significant bottleneck in decoding time . |
| Approach: | They propose to eliminate some of the retrieved information that might not contribute essential information to the answer generation process. |
| Outcome: | The proposed method reduces run-time by up to 62.2% with only 2% reduction in performance and improves performance. |
Towards Adaptive Memory-Based Optimization for Enhanced Retrieval-Augmented Generation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for enhancing response accuracy and accuracy struggle with open-domain QA tasks because they perform independent retrieval operations without maintaining a summarizing memory or using adaptive retrieval strategies. |
| Approach: | They propose a method that integrates non-parametric knowledge from external knowledge bases into models to enhance response accuracy while mitigating factual errors and hallucinations. |
| Outcome: | The proposed method improves on open-domain QA datasets and reduces noise and hallucinations due to redundant information and insufficient information integration. |
AttnComp: Attention-Guided Adaptive Context Compression for Retrieval-Augmented Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for augmented large language models suffer from irrelevant retrieved content . existing methods struggle to adapt compression rates for different context, maintain low latency . |
| Approach: | We propose an adaptive, efficient and context-aware compression framework to reduce retrieved content . AttnComp uses a top-p compression algorithm to retain the minimal set of documents whose attention weights exceed a threshold. |
| Outcome: | Experiments show that AttnComp outperforms existing compression methods and uncompressed baselines in achieving higher accuracy with substantial compression rates and lower latency. |