Papers with Self-RAG
Open-RAG: Enhanced Retrieval Augmented Reasoning with Open-Source Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to integrate Large Language Models with external knowledge suffer from limited reasoning capabilities, especially when using open-source LLMs. |
| Approach: | They propose a framework that transforms an arbitrary dense LLM into a parameter-efficient sparse mixture of experts (MoE) model capable of handling complex reasoning tasks. |
| Outcome: | The proposed framework transforms an arbitrary dense LLM into a parameter-efficient sparse mixture of experts (MoE) model capable of handling complex reasoning tasks, including both single- and multi-hop queries. |
PROGRAM: Programmatic Retrieval Optimization with Generative Reasoning and Augmented Multi-queries (2026.findings-acl)
Copied to clipboard
| Challenge: | Current retrieval-augmented generation methods struggle with complex multi-hop reasoning, relying on unstructured semantic matching that lacks the logical structure needed to systematically guide retrieval. |
| Approach: | They propose a framework that elevates retrieval to structured, program-guided reasoning by combining three stages of program-type selection and evidence accumulation. |
| Outcome: | Evaluated on five benchmarks including HotPotQA, 2WikiMultihopQA, ARC-Challenge, MMLU-Pro, and MedQA with various LLMs, PROGRAM achieves state-of-the-art performance with up to 24% relative improvement on HotPtQA and 13.2% on MedQA over strong baselines including FLARE, ProbTree and Self-RAG. |