AI Knowledge Assist: An Automated Approach for the Creation of Knowledge Bases for Conversational AI Agents (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing knowledge base is time-consuming and deters the adoption of conversational AI systems in contact centers. |
| Approach: | They propose a system that extracts knowledge in the form of question-answer (QA) pairs from historical customeragent conversations to automatically build a knowledge base. |
| Outcome: | The proposed system outperforms larger closed-source LLMs on internal data and achieves above 90% accuracy in answering informationseeking questions. |
Similar Papers
Optimizing Retrieval-Augmented Generation for E-Commerce How-To Assistance (2026.acl-industry)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have significantly improved Retrieval-Augmented Generation (RAG), enabling assistants that can reliably ground responses in external knowledge sources while maintaining high-quality natural language interaction. |
| Approach: | They propose a RAG-based How-To Assistant that groundes responses in a proprietary knowledge base to provide personalized customer support. |
| Outcome: | The proposed assistant can ground responses in a proprietary knowledge base while maintaining high-quality natural language interaction. |
Bringing the State-of-the-Art to Customers: A Neural Agent Assistant Framework for Customer Service Support (2022.emnlp-industry)
Copied to clipboard
Stephen Obadinma, Faiza Khan Khattak, Shirley Wang, Tania Sidhorn, Elaine Lau, Sean Robertson, Jingcheng Niu, Winnie Au, Alif Munim, Karthik Raja Kalaiselvi Bhaskar
| Challenge: | Creating agent assistants that can help improve customer service support requires inputs from industry users and their customers as well as knowledge of state-of-the-art natural language processing (NLP) technology. |
| Approach: | They propose to combine expertise from academia and industry to build task/domain-specific Neural Agent Assistants with three high-level components for: (1) Intent Identification, (2) Context Retrieval, and (3) Response Generation. |
| Outcome: | The proposed framework is based on three case studies of industry partners who successfully adapt the framework to their unique challenges. |
DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text (2024.findings-naacl)
Copied to clipboard
| Challenge: | Retrievalaugmented LLMs have been used to ground LLM in external knowledge . a gap exists in the current landscape regarding the effectiveness of grounding LLM on heterogeneous knowledge sources. |
| Approach: | They propose a model that uses symbolic language to generate symbolic queries . they use a dataset that is generated using predefined reasoning chains and human annotation . |
| Outcome: | The proposed model outperforms previous approaches by a significant margin in QA tasks over text. |
Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent Large Language Models (LLMs) generate factually incorrect answers based on their parametric memory. |
| Approach: | They propose a retrieval-augmented large language model that can dynamically select the most suitable strategy based on query complexity. |
| Outcome: | The proposed approach improves the performance of QA systems on open-domain QA datasets. |
It’s Not under the Lamppost: Expanding the Reach of Conversational AI (2024.lrec-main)
Copied to clipboard
| Challenge: | Focused probes into the capabilities of language-based assistants easily reveal significant areas of brittleness that demonstrate large gaps in their coverage. |
| Approach: | They propose a process for collecting specific kinds of data to uncover these gaps and an annotation scheme for system responses. |
| Outcome: | The proposed system includes both Conventional and GenAI systems, including ChatGPT and Bard/Gemini. |
Adaptive Retrieval-Augmented Generation for Conversational Systems (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies have shown the effectiveness of retrieving and augmenting external knowledge for informative responses. |
| Approach: | They propose to use a gating model to predict if a conversational system requires retrieval-augmented generation to generate high-quality responses with high confidence. |
| Outcome: | The proposed model can predict if a conversational system requires RAG to generate high-quality responses with high confidence. |
Can Generative Pre-trained Language Models Serve As Knowledge Bases for Closed-book QA? (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work is limited in using small benchmarks with high test-train overlaps. |
| Approach: | They construct a dataset of closed-book QA using SQuAD and investigate the performance of BART. |
| Outcome: | Experiments show that pre-trained language models can achieve high performance on closed-book QA tasks. |
Agent Assist through Conversation Analysis (2020.emnlp-demos)
Copied to clipboard
Kshitij Fadnis, Nathaniel Mills, Jatin Ganhotra, Haggai Roitman, Gaurav Pandey, Doron Cohen, Yosi Mass, Shai Erera, Chulaka Gunasekara, Danish Contractor, Siva Patel, Q. Vera Liao, Sachindra Joshi, Luis Lastras, David Konopnicki
| Challenge: | Using conversational approach to information retrieval for agent assistance, customer support agents are a critical part of an organization's customer support team. |
| Approach: | They propose a conversational approach to information retrieval for agent assistance that monitors an evolving conversation and recommends both responses and URLs of documents. |
| Outcome: | The proposed system monitors an evolving conversation and recommends both responses and URLs of documents the agent can use in replies to their client. |
Retrieval Enhancements for RAG: Insights from a Deployed Customer Support Chatbot (2026.eacl-industry)
Copied to clipboard
Daniel González Juclà, Mohit Tuteja, Marcos Esteve Casademunt, Keshav Unnikrishnan, Yasir Usmani, Arvind Roshaan
| Challenge: | a persistent gap remains between Recall@10 and Recall @50 across datasets . |
| Approach: | They evaluate embedding model comparison, Reciprocal Rank Fusion and embedded concatenation techniques to improve retrieval quality. |
| Outcome: | The proposed methods outperform traditional cross-encoders in identifying high-relevance passages. |
Towards Reliable Agents: Benchmarking Customized LLM-Based Retrieval-Augmented Generation Frameworks with Deployment Validation (2025.naacl-industry)
Copied to clipboard
| Challenge: | Existing benchmarks for general-purpose RAG systems, such as CRAG, RGB, MultiHop-RAG, and CRUD-RAGG, are limited and lack a benchmark specifically tailored to evaluate frameworks. |
| Approach: | They evaluated OpenAI’s Assistants API versus a RAG assistant built with Langchain and deployed a system based on benchmark insights as a course assistant over a two-year span. |
| Outcome: | The proposed benchmarks show that domain-specific retrieval impacts response accuracy and highlight key challenges in real-world deployment. |