StruNRAG: Evaluation of OCR-Induced Structural Noise on RAG Robustness (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluations of RAG systems ignore structural noise, authors say . complex layouts can cause OCR failures and disrupt semantic flow of text . advanced LLMs demonstrate robustness against local noise, but struggle to maintain reasoning capabilities under severe structural disruption that fragments global context. |
| Approach: | They propose a benchmark to evaluate RAG robustness against OCR-induced structural perturbations. |
| Outcome: | The proposed benchmark systematically injects three categories of real-world structural noise into a bilingual dataset of 2,132 question-answer pairs . results show that advanced LLMs demonstrate robustness against local noise, but struggle to maintain reasoning capabilities under severe structural disruption . |
Similar Papers
When Good OCR Is Not Enough: Benchmarking OCR Robustness for Retrieval-Augmented Generation (2026.acl-industry)
Copied to clipboard
Lin Sun, null Wangdexian, Jingang Huang, Linglin Zhang, Change Jia, Zhengwei Cheng, Xiangzheng Zhang
| Challenge: | Existing OCR benchmarks rely on character-level metrics to measure downstream performance . high OCR accuracy does not translate into strong downstream performance, authors say . |
| Approach: | They propose an OCR benchmark for industrial RAG systems that measures character-level metrics . they find that high OCR accuracy does not translate into strong downstream RAG performance . |
| Outcome: | The proposed benchmark shows that high OCR accuracy does not translate into strong downstream performance . structural and semantic errors can cause substantial retrieval failures even when WER/CER remains low. |
Enhancing Noise Robustness of Retrieval-Augmented Language Models with Adaptive Adversarial Training (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) exhibit substantial capabilities yet face challenges such as hallucination, outdated knowledge, and untraceable reasoning processes. |
| Approach: | They propose a retrieval-augmented generation approach that leverages adaptive adversarial training to dynamically adjust the model’s training process in response to retrieval noises. |
| Outcome: | The proposed approach improves the performance of the LLaMA-2 7B model under diverse noise conditions. |
LLM-Generated Text May Harm Your Retrieval! A Robust Detection Strategy for Retrieval-Augmented Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Retrieval-augmented generation (RAG) improves accuracy and timeliness of large language models, but external corpora may become contaminated with LLM-generated texts. |
| Approach: | They propose a method that integrates external knowledge retrieved from external sources into RAG to filter out LLM-generated texts from retrieved results. |
| Outcome: | The proposed method mitigates performance degradation and improves stability of RAG systems. |
Typos that Broke the RAG’s Back: Genetic Attack on RAG Pipeline by Simulating Documents in the Wild via Low-level Perturbations (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on the robustness of Large Language Models (LLMs) overlook the interconnected relationships between RAG components or the potential threats prevalent in real-world databases, such as minor textual errors. |
| Approach: | They propose a novel attack method that exploits vulnerabilities in RAG components and tests its robustness against noisy documents. |
| Outcome: | The proposed method devastates the performance of each component and their synergy, and significantly devases the performance. |
ParetoRAG: Leveraging Sentence-Context Attention for Robust and Efficient Retrieval-Augmented Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Retrieval-augmented generation systems face persistent challenges in retrieval inefficiency and the inability of LLMs to filter out irrelevant information. |
| Approach: | They propose an unsupervised framework that optimizes RAG systems through sentence-level refinement guided by the Pareto principle. |
| Outcome: | The proposed framework achieves dual improvements in retrieval precision and generation quality without additional training or API resources while using only 40% of the tokens compared to traditional approaches. |
Equipping Retrieval-Augmented Large Language Models with Document Structure Awareness (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to retrieval-augmented generation ignore valuable structure that is crucial for document organization. |
| Approach: | They propose a framework that explicitly incorporates structural information throughout the RAG process. |
| Outcome: | The proposed framework incorporates structural information throughout the RAG process. |
Out of Style: RAG’s Fragility to Linguistic Variation (2026.eacl-long)
Copied to clipboard
| Challenge: | linguistic reformulations impact both retrieval and generation stages, leading to a relative performance drop of up to 40.41% for less formal queries and 38.86% for queries containing grammatical errors. |
| Approach: | They evaluate two retrieval models and nine LLMs across four QA datasets and examine how linguistic reformulations impact RAG performance. |
| Outcome: | The proposed models show that linguistic reformulations significantly impact both retrieval and generation stages, leading to a performance drop of up to 40.41% for less formal queries and 38.86% for queries containing grammatical errors. |
Stable-RAG: Mitigating Retrieval-Permutation-Induced Hallucinations in Retrieval-Augmented Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing RAG methods focus on enhancing LLM robustness to low-quality retrieval, but neither address permutation sensitivity. |
| Approach: | They propose a method that exploits permutation sensitivity to mitigate hallucinations in Large Language Models. |
| Outcome: | The proposed model improves answer accuracy, reasoning consistency, and generalization across datasets, retrievers, and input lengths compared with strong baselines. |
LONG2RAG: Evaluating Long-Context & Long-Form Retrieval-Augmented Generation with Key Point Recall (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Retrieval-augmented generation (RAG) is a promising approach to address limitations of fixed knowledge in large language models. |
| Approach: | They propose a benchmark and a metric to assess LLMs' ability to generate long-form responses that exploit retrieved information. |
| Outcome: | The proposed benchmarks lack a comprehensive evaluation method to assess LLMs' ability to generate long-form responses that effectively exploits retrieved information. |
ReflectiveRAG: Rethinking Adaptivity in Retrieval-Augmented Generation (2026.eacl-industry)
Copied to clipboard
| Challenge: | Existing methods for retrieval-augmented generation (RAG) fail to assess evidence sufficiency, detect subtle mismatches or reduce redundancy. |
| Approach: | They propose a lightweight yet reasoning-driven architecture that enhances factual grounding . ReflectiveRAG employs self-reflective retrieval and Contrastive noise removal . |
| Outcome: | a new architecture improves factual grounding by using self-reflective retrieval and Contrastive noise removal. |