Intelligent Predictive Maintenance RAG framework for Power Plants: Enhancing QA with StyleDFS and Domain Specific Instruction Tuning (2024.emnlp-industry)
Copied to clipboard
Seongtae Hong, Joong Shin, Jaehyung Seo, Taemin Lee, Jeongbae Park, Cho Young, Byeongho Choi, Heuiseok Lim
| Challenge: | Existing off-premise Question-Answering systems based on Large Language Models face data leakage and domain-specific tuning challenges. |
| Approach: | They propose an on-premise intelligent PMS framework based on a chunking method . they propose instruction tuning using relevant domain-specific data improves LLM performance . |
| Outcome: | The proposed framework improves performance even under limited data conditions. |
Similar Papers
RAG4ITOps: A Supervised Fine-Tunable and Comprehensive RAG Framework for IT Operations and Maintenance (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have improved the open-domain QA’s performance, but how to efficiently handle enterprise-exclusive corpora and build domain-specific QA systems are still not studied for industrial applications. |
| Approach: | They propose a general and comprehensive framework based on Retrieval Augmented Generation (RAG) and facilitate the whole business process of establishing QA systems for IT operations and maintenance. |
| Outcome: | The proposed framework achieves superior results on two kinds of QA tasks. |
RAGulator: Lightweight Out-of-Context Detectors for Grounded Text Generation (2025.emnlp-industry)
Copied to clipboard
| Challenge: | In enterprise settings, Generative AI has received widespread adoption as a tool to uplift employees' productivity. |
| Approach: | They develop lightweight models capable of detecting when LLM-generated text deviates from retrieved source documents semantically. |
| Outcome: | The proposed models outperform open-source alternatives on credit policy and sustainability reports used in the banking industry. |
DA-Pred: Performance Prediction for Text Summarization under Domain-Shift and Instruct-Tuning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) often don’t perform as expected under Domain Shift or after Instruct-tuning. |
| Approach: | They propose a method that uses the known performance in high-resource domains and fine-tuning settings to predict performance in low-resourced domains or base models. |
| Outcome: | The proposed method can help researchers decide if resources should be allocated for data labeling and LLM Instruct-tuning. |
Fine-Grained Data Ordering Improves Fine-Tuning for Large Language Models (2026.findings-acl)
Copied to clipboard
Xiaomeng Hu, Yixuan Tang, Haoze Li, Hao Chen, Qi Zhang, Zhanming Shen, Yiming Zhang, Haobo Wang, Junbo Zhao
| Challenge: | Prior work focused on data preprocessing, focusing on filtering and cleaning data . a study aimed to improve fine-grained scheduling of data order in epochs . |
| Approach: | They propose a fine-grained scheduling method of data order in epochs to fill this gap . they define data difficulty based on relevance between data and model . |
| Outcome: | The proposed method improves on pre-training and small-scale fine-tuning experiments 2.4% over baselines. |
SCAR: Data Selection via Style Consistency-Aware Response Ranking for Efficient Instruction-Tuning of Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that manually ensuring a consistent response style and maintaining high data quality can significantly improve the performance of fine-tuned Large Language Models (LLMs). |
| Approach: | They introduce a style-aware response ranking system that prioritizes instruction-response pairs based on their stylistic consistency. |
| Outcome: | The proposed model matches or surpasses models trained on the entire dataset in coding and open-ended question-answering benchmarks. |
Retrieval Enhancements for RAG: Insights from a Deployed Customer Support Chatbot (2026.eacl-industry)
Copied to clipboard
Daniel González Juclà, Mohit Tuteja, Marcos Esteve Casademunt, Keshav Unnikrishnan, Yasir Usmani, Arvind Roshaan
| Challenge: | a persistent gap remains between Recall@10 and Recall @50 across datasets . |
| Approach: | They evaluate embedding model comparison, Reciprocal Rank Fusion and embedded concatenation techniques to improve retrieval quality. |
| Outcome: | The proposed methods outperform traditional cross-encoders in identifying high-relevance passages. |
MES-RAG: Bringing Multi-modal, Entity-Storage, and Secure Enhancements to RAG (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to retrieve entity information are limited by document level retrieval and intermingled storage of information from different entities. |
| Approach: | They propose a framework that enhances entity-specific query handling . MES-RAG introduces proactive security measures that ensure system integrity . |
| Outcome: | Experimental results show that MES-RAG improves accuracy and recall . the framework can be integrated into existing RAG architectures . |
MAIN-RAG: Multi-Agent Filtering Retrieval-Augmented Generation (2025.acl-long)
Copied to clipboard
Chia-Yuan Chang, Zhimeng Jiang, Vineeth Rakesh, Menghai Pan, Chin-Chia Michael Yeh, Guanchu Wang, Mingzhi Hu, Zhichao Xu, Yan Zheng, Mahashweta Das, Na Zou
| Challenge: | Existing RAG systems struggle with the quality of retrieval documents, causing performance degradation and reducing performance. |
| Approach: | They propose a training-free RAG framework that leverages multiple LLM agents to collaboratively filter and score retrieved documents. |
| Outcome: | The proposed framework outperforms existing RAG frameworks in QA benchmarks and shows superior answer consistency and answer accuracy over baseline methods. |
Know Your RAG: Dataset Taxonomy and Generation Strategies for Evaluating RAG Systems (2025.coling-industry)
Copied to clipboard
Rafael Teixeira de Lima, Shubham Gupta, Cesar Berrospi Ramis, Lokesh Mishra, Michele Dolfi, Peter Staar, Panagiotis Vagenas
| Challenge: | Retrieval Augmented Generation (RAG) systems are widespread in the industry. |
| Approach: | They propose to use Q&A datasets to assess retrieval performance and label-targeted data generation to refine RAG datasets. |
| Outcome: | The proposed system can generate Q&A datasets with fine-tuned small LLMs. |
Smarter, not Bigger: Fine-Tuned RAG-Enhanced LLMs for Automotive Hardware-in-the-Loop Testing (2026.acl-industry)
Copied to clipboard
| Challenge: | Hardware-in-the-Loop (HIL) testing is essential for automotive validation but suffers from fragmented and underutilized test artifacts. |
| Approach: | They propose to integrate semantic retrieval with domain-adapted large language models to support test engineers in real-world HIL workflows. |
| Outcome: | The proposed system improves perceived helpfulness, truthfulness, and satisfaction over general-purpose LLMs. |