Evaluating the Performance of RAG Methods for Conversational AI in the Airport Domain (2025.naacl-industry)
Copied to clipboard
| Challenge: | a conversational AI system that answers standard airport queries and resolves airport terminology is ideal for airports from the top 20 in terms of annual passenger numbers. |
| Approach: | They propose a Conversational AI system that enables staff to communicate with flight systems . the system answers standard airport queries and resolves airport terminology . |
| Outcome: | The proposed system answers standard airport queries and resolves airport terminology, jargon, abbreviations and dynamic questions involving reasoning. |
Similar Papers
Comprehensive Comparison of RAG Methods Across Multi-Domain Conversational QA (2026.eacl-srw)
Copied to clipboard
| Challenge: | Existing studies evaluate RAG methods in isolation and focus on single-turn settings. |
| Approach: | They compare retrieval-augmented generation methods for multi-turn conversational QA with those that use dialogue history and coreference to ground large language models. |
| Outcome: | The proposed methods outperform vanilla RAG and advanced methods fail to yield gains and can even degrade performance below the No-RAG baseline. |
A Survey of RAG-Reasoning Systems in Large Language Models (2025.findings-emnlp)
Copied to clipboard
Yangning Li, Weizhi Zhang, Yuyao Yang, Wei-Chieh Huang, Yaozu Wu, Junyu Luo, Yuanchen Bei, Henry Peng Zou, Xiao Luo, Yusheng Zhao, Chunkit Chan, Yankai Chen, Zhongfen Deng, Yinghui Li, Hai-Tao Zheng, Dongyuan Li, Renhe Jiang, Ming Zhang, Yangqiu Song, Philip S. Yu
| Challenge: | a survey of RAG-based reasoning-based approaches shows that it is not effective for multi-step inferences. |
| Approach: | They map how advanced reasoning optimizes each stage of RAG . they show how retrieved knowledge supply missing premises and expand context for complex inference . |
| Outcome: | The proposed frameworks achieve state-of-the-art across knowledge-intensive benchmarks. |
Adaptive Retrieval-Augmented Generation for Conversational Systems (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies have shown the effectiveness of retrieving and augmenting external knowledge for informative responses. |
| Approach: | They propose to use a gating model to predict if a conversational system requires retrieval-augmented generation to generate high-quality responses with high confidence. |
| Outcome: | The proposed model can predict if a conversational system requires RAG to generate high-quality responses with high confidence. |
RAG over Tables: Hierarchical Memory Index, Multi-Stage Retrieval, and Benchmarking (2026.findings-acl)
Copied to clipboard
| Challenge: | Retrieval-Augmented Generation (RAG) integrates knowledge from tables with an external knowledge base to improve the answer relevance and accuracy. |
| Approach: | They propose a table-corpora-aware RAG framework called T-RAG to integrate external knowledge into Large Language Models (LLMs) they then develop a multi-table question answering benchmark called MultiTableQA which spans 3 different task types, 57,193 tables, and 23,758 questions in total. |
| Outcome: | The proposed framework achieves state-of-the-art accuracy, recall, and runtime performance, with improvements of up to 9.4%. |
Searching for Best Practices in Retrieval-Augmented Generation (2024.emnlp-main)
Copied to clipboard
Xiaohua Wang, Zhenghua Wang, Xuan Gao, Feiran Zhang, Yixin Wu, Zhibo Xu, Tianyuan Shi, Zhengyuan Wang, Shizheng Li, Qi Qian, Ruicheng Yin, Changze Lv, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Retrieval-augmented generation (RAG) techniques have proven to be effective in integrating up-to-date information, mitigating hallucinations, and enhancing response quality, especially in specialized domains. |
| Approach: | They propose several strategies for deploying RAG that balance performance and efficiency. |
| Outcome: | The proposed approaches can significantly enhance question-answering capabilities and accelerate the generation of multimodal content using a “retrieval as generation” strategy. |
CORAL: Benchmarking Multi-turn Conversational Retrieval-Augmented Generation (2025.findings-naacl)
Copied to clipboard
Yiruo Cheng, Kelong Mao, Ziliang Zhao, Guanting Dong, Hongjin Qian, Yongkang Wu, Tetsuya Sakai, Ji-Rong Wen, Zhicheng Dou
| Challenge: | Existing research focuses on single-turn RAG, leaving a gap in addressing multi-turn conversations . a new benchmark is designed to assess RAG systems in realistic multi-turned conversations based on Wikipedia . |
| Approach: | They propose a large-scale benchmark to assess RAG systems in multi-turn contexts . CORAL includes diverse information-seeking conversations automatically derived from Wikipedia . authors propose unified framework to standardize various conversational RAG methods . |
| Outcome: | The proposed framework supports three core tasks of conversational RAG: passage retrieval, response generation, and citation labeling. |
Know Your RAG: Dataset Taxonomy and Generation Strategies for Evaluating RAG Systems (2025.coling-industry)
Copied to clipboard
Rafael Teixeira de Lima, Shubham Gupta, Cesar Berrospi Ramis, Lokesh Mishra, Michele Dolfi, Peter Staar, Panagiotis Vagenas
| Challenge: | Retrieval Augmented Generation (RAG) systems are widespread in the industry. |
| Approach: | They propose to use Q&A datasets to assess retrieval performance and label-targeted data generation to refine RAG datasets. |
| Outcome: | The proposed system can generate Q&A datasets with fine-tuned small LLMs. |
RAGLAB: A Modular and Research-Oriented Unified Framework for Retrieval-Augmented Generation (2024.emnlp-demo)
Copied to clipboard
Xuanwang Zhang, Yun-Ze Song, Yidong Wang, Shuyun Tang, Xinfeng Li, Zhengran Zeng, Zhen Wu, Wei Ye, Wenyuan Xu, Yue Zhang, Xinyu Dai, Shikun Zhang, Qingsong Wen
| Challenge: | Existing research on Retrieval Augmented Generation (RAG) does not address the problem of hallucinations and real-time updating of knowledge. |
| Approach: | They propose a modular open-source library to equip LLMs with external knowledge. |
| Outcome: | The proposed approach reduces the need for expensive open-source tools and lacks fair comparisons between novel RAG algorithms. |
MS-RAG: Simple and Effective Multi-Semantic Retrieval-Augmented Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for large language models suffer from poor indexing and inference speed . graph-based RAGs heavily rely on LLM for retrieval thus inference slow . |
| Approach: | They propose retrieval-augmented generation (RAG) which integrates knowledge with dense vectors to build a multi-semantic RAG. |
| Outcome: | The proposed method achieves state-of-the-art performance with faster inference speed compared to existing methods . |
PanoramaRAG: Enabling Consistent Global Topic Awareness in Graph-Based RAG (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing graph-based methods for enhancing Large Language Models (LLMs) with external knowledge are focusing on local relationships, resulting in suboptimal performance for tasks that require global context. |
| Approach: | They propose a "panorama"-guided paradigm that integrates a light yet comprehensive "panoramic" of the corpus to guide all stages of the retrieval process. |
| Outcome: | The proposed paradigm performs well across five datasets and a variety of tasks. |