Harmonizing Diverse Models: A Layer-wise Merging Strategy for Consistent Generation (2025.emnlp-industry)
Copied to clipboard
| Challenge: | RAG systems often generate inconsistent outputs for semantically equivalent inputs . this unpredictability undermines the reliability of RAG and poses challenges for adoption in high-stakes or knowledge-sensitive domains such as finance, healthcare, and scientific research. |
| Approach: | They propose a method that integrates knowledge from specialized models into a single model to improve output consistency. |
| Outcome: | The proposed model significantly improves output consistency, achieving approximately 47.5% improvement in response similarity over baseline. |
Similar Papers
RAG in the Wild: On the (In)effectiveness of LLMs with Mixture-of-Knowledge Retrieval Augmentation (2026.findings-acl)
Copied to clipboard
| Challenge: | Retrieval-augmented generation (RAG) enhances large language models by integrating external knowledge retrieved at inference time. |
| Approach: | They evaluate RAG systems using MassiveDS, a large-scale datastore with mixture of knowledge. |
| Outcome: | The proposed approach improves performance on knowledge-intensive NLP tasks. |
Enhancing Retrieval-Augmented Generation: A Study of Best Practices (2025.coling-main)
Copied to clipboard
| Challenge: | Retrieval-augmented generation systems have shown remarkable advancements by integrating retrieval mechanisms into language models, enhancing their ability to produce more accurate and contextually relevant responses. |
| Approach: | They propose to integrate query expansion, various novel retrieval strategies, and a Contrastive In-Context Learning RAG to improve response quality. |
| Outcome: | The proposed RAGs incorporate query expansion, various novel retrieval strategies, and a novel Contrastive In-Context Learning RAG. |
Systematic Knowledge Injection into Large Language Models via Diverse Augmentation for Domain-Specific RAG (2025.findings-naacl)
Copied to clipboard
Kushagra Bhushan, Yatin Nandwani, Dinesh Khandelwal, Sonam Gupta, Gaurav Pandey, Dinesh Raghu, Sachindra Joshi
| Challenge: | Retrieval-Augmented Generation (RAG) enhances response relevance by incorporating retrieved domain knowledge in the context, retrieval errors can still lead to hallucinations and incorrect answers. |
| Approach: | They propose a framework that augments the learning process by context augmentation and knowledge paraphrasing by incorporating retrieved domain knowledge into the context. |
| Outcome: | The proposed framework achieves 10% relative gain in token-level recall while preserving the LLM’s generalization capabilities. |
M-RAG: Reinforcing Large Language Model Performance through Retrieval-Augmented Generation with Multiple Partitions (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for retrieving relevant memories from an external database are coarse-grained and can cause noise and focus on crucial memories. |
| Approach: | They propose a multiple partition paradigm for RAG where each database partition serves as a basic unit for execution. |
| Outcome: | The proposed framework outperforms baseline methods on three language generation tasks on seven datasets. |
AED-RAG: Continuous Multi-Granular Context Fusion for Retrieval-Augmented Generation via Adaptive Ensemble Decoding (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing alignment strategies that rely on discrete reranking struggle to address this granularity mismatch or effectively balance external evidence with internal knowledge. |
| Approach: | They propose a framework that synergizes discrete retrieval with continuous reranking to discern the information density differences between unstructured narrative passages and structured knowledge triplets. |
| Outcome: | Extensive experiments on four open-domain QA benchmarks show that AED-RAG significantly outperforms competitive baselines. |
Retrieval-augmented Generation across Heterogeneous Knowledge (2022.naacl-srw)
Copied to clipboard
| Challenge: | Existing methods for retrieving knowledge from a single source homogeneous corpus have been gaining increasing attention in the field of natural language processing (NLP) however, they still suffer from the following drawbacks: (i) They are usually trained offline, making the model agnostic to the latest information, e.g., asking a chat-bot about COVID-19. |
| Approach: | They propose to use a single-source homogeneous corpus to generate retrieval-augmented generation models that can learn from the pre-training corpus. |
| Outcome: | The proposed methods have been applied to various knowledge-intensive NLP tasks, but most of the work has focused on retrieving unstructured text documents from Wikipedia. |
RAG+: Enhancing Retrieval-Augmented Generation with Application-Aware Reasoning (2025.emnlp-main)
Copied to clipboard
Yu Wang, Shiwan Zhao, Zhihu Wang, Ming Fan, Xicheng Zhang, Yubo Zhang, Zhengfan Wang, Heyuan Huang, Ting Liu
| Challenge: | Existing RAG paradigms often overlook the cognitive step of applying knowledge, leaving a gap between retrieved facts and task-specific reasoning. |
| Approach: | They introduce a module extension that integrates application-aware reasoning into the RAG pipeline. |
| Outcome: | Experiments show that RAG+ outperforms standard RAG variants and achieves gains of 3–5% in complex scenarios. |
RAG-Studio: Towards In-Domain Adaptation of Retrieval Augmented Generation Through Self-Alignment (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing RAG systems that use pre-trained LLMs and retrievers often fail in specialized domains and applications. |
| Approach: | They propose a self-aligned training framework that adapts general RAG models to specific domains solely through synthetic data. |
| Outcome: | Experiments on specialized domain corpus, general LLM, and general retriever show that the self-aligned training framework outperforms human-annotated training data in specialized fields. |
RAG-Instruct: Boosting LLMs with Diverse Retrieval-Augmented Instructions (2025.emnlp-main)
Copied to clipboard
| Challenge: | Retrieval-Augmented Generation (RAG) has emerged as a key paradigm for enhancing large language models by incorporating external knowledge. |
| Approach: | They propose a method for synthesizing diverse and high-quality RAG instruction data based on any source corpus. |
| Outcome: | The proposed method outperforms existing methods in multiple tasks and achieves strong zero-shot performance. |
Searching for Best Practices in Retrieval-Augmented Generation (2024.emnlp-main)
Copied to clipboard
Xiaohua Wang, Zhenghua Wang, Xuan Gao, Feiran Zhang, Yixin Wu, Zhibo Xu, Tianyuan Shi, Zhengyuan Wang, Shizheng Li, Qi Qian, Ruicheng Yin, Changze Lv, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Retrieval-augmented generation (RAG) techniques have proven to be effective in integrating up-to-date information, mitigating hallucinations, and enhancing response quality, especially in specialized domains. |
| Approach: | They propose several strategies for deploying RAG that balance performance and efficiency. |
| Outcome: | The proposed approaches can significantly enhance question-answering capabilities and accelerate the generation of multimodal content using a “retrieval as generation” strategy. |