X-Router: Decoupling Knowledge and Reasoning for Cost-Effective LLM Inference (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing adaptive methods focus on a single axis, overlooking evidence need and reasoning depth are only partially correlated. |
| Approach: | They propose a dual-axis routing framework that separates retrieval necessity from reasoning necessity under a user-defined cost–quality trade-off. |
| Outcome: | The proposed framework reduces token usage and latency while improving answer quality over strong baselines. |
Similar Papers
CoT-RAG: Integrating Chain of Thought and Retrieval-Augmented Generation to Enhance Reasoning in Large Language Models (2025.findings-emnlp)
Copied to clipboard
Feiyang Li, Peng Fang, Zhan Shi, Arijit Khan, Fang Wang, Weihao Wang, null Zhangxin-hw, Cui Yongjian
| Challenge: | Chain-of-thought reasoning has two key limitations: lack of reliability when solely relying on LLM-generated reasoning chains and interference from natural language reasoning steps with the models’ inference logic. |
| Approach: | They propose a chain-of-thought reasoning framework with three key designs to address these issues. |
| Outcome: | The proposed framework improves the performance of large language models on complex tasks by incorporating knowledge graphs and learnable knowledge case-aware RAG. |
Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generation (2025.naacl-long)
Copied to clipboard
Satyapriya Krishna, Kalpesh Krishna, Anhad Mohananey, Steven Schwarcz, Adam Stambler, Shyam Upadhyay, Manaal Faruqui
| Challenge: | Recent advances in Large Language Models (LLMs) have significantly enhanced their capabilities across various cognitive tasks. |
| Approach: | They propose a high-quality evaluation dataset to test LLMs' ability to provide factual responses, assess retrieval capabilities, and evaluate the reasoning required to generate final answers. |
| Outcome: | The proposed framework improves performance in end-to-end RAG scenarios. |
SATER: A Self-Aware and Token-Efficient Approach to Routing and Cascading (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) demonstrate remarkable performance across diverse tasks, yet their effectiveness often depends on costly commercial APIs or cloud services. |
| Approach: | They propose a dual-mode compatible approach that fine-tunes models through shortest-response preference optimization and a confidence-aware rejection mechanism. |
| Outcome: | The proposed approach reduces redundant outputs and response times while reducing computational costs by over 50% and cascade latency by over 80%. |
SkewRoute: Training-Free LLM Routing for Knowledge Graph Retrieval-Augmented Generation via Score Skewness of Retrieved Context (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models incur high inference costs during deployment, causing hallucination . no dedicated routing methods exist for RAG, and existing training-based routers face challenges scaling to this domain . |
| Approach: | They propose a plug-and-play routing framework that optimizes performance and cost . the framework delivers over 3x higher routing effectiveness while reducing runtime to less than 0.001x . |
| Outcome: | The proposed framework delivers over 3x higher routing effectiveness while reducing runtime to less than 0.001x compared to existing methods. |
TensorOpera Router: A Multi-Model Router for Efficient LLM Inference (2024.emnlp-industry)
Copied to clipboard
Dimitris Stripelis, Zhaozhuo Xu, Zijian Hu, Alay Shah, Han Jin, Yuhang Yao, Jipeng Zhang, Tong Zhang, Salman Avestimehr, Chaoyang He
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable performance across a diverse set of domain-specific tasks. |
| Approach: | They propose a non-monolithic LLM querying system that seamlessly integrates various LLM experts into a single query interface and dynamically routes incoming queries to the most high-performant expert based on query’s requirements. |
| Outcome: | The proposed model improves query efficiency by 40% and costs by 30% while maintaining or enhancing model performance by 10%. |
DeepSieve: Information Sieving via LLM-as-a-Knowledge-Router (2026.findings-eacl)
Copied to clipboard
Minghao Guo, Qingcheng Zeng, Xujiang Zhao, Yanchi Liu, Wenchao Yu, Mengnan Du, Haifeng Chen, Wei Cheng
| Challenge: | Existing RAG methods lack fine-grained control over query and source sides, resulting in noisy retrieval and shallow reasoning. |
| Approach: | They propose an agentic RAG framework that integrates information sieving via LLM-as-a-knowledge-router. |
| Outcome: | Experiments on multi-hop QA tasks across heterogeneous sources demonstrate improved reasoning depth, retrieval precision, and interpretability over conventional approaches. |
Open-RAG: Enhanced Retrieval Augmented Reasoning with Open-Source Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to integrate Large Language Models with external knowledge suffer from limited reasoning capabilities, especially when using open-source LLMs. |
| Approach: | They propose a framework that transforms an arbitrary dense LLM into a parameter-efficient sparse mixture of experts (MoE) model capable of handling complex reasoning tasks. |
| Outcome: | The proposed framework transforms an arbitrary dense LLM into a parameter-efficient sparse mixture of experts (MoE) model capable of handling complex reasoning tasks, including both single- and multi-hop queries. |
StitchLLM: Serving LLMs, One Block at a Time (2025.acl-long)
Copied to clipboard
Bodun Hu, Shuozhe Li, Saurabh Agarwal, Myungjin Lee, Akshay Jajoo, Jiamin Li, Le Xu, Geon-Woo Kim, Donghyun Kim, Hong Xu, Amy Zhang, Aditya Akella
| Challenge: | Existing techniques like distillation and pruning are not efficient for large language models. |
| Approach: | They propose a dynamic model routing framework that uses a powerful bottom model to process all queries and a lightweight routing mechanism to allocate computational resources appropriately. |
| Outcome: | The proposed framework improves system throughput while minimizing performance degradation. |
BiCSRouter: Bi-Level Cross-System Routing for Utility-Aware LLM Inference (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing routing frameworks operate within a single computational paradigm . a cross-system routing framework that integrates two orthogonal regimes is proposed . |
| Approach: | They propose a cross-system routing framework that integrates two orthogonal regimes . they propose MBPP-based model that decomposes routing into intra-regime configuration selection and inter-regem system selection . |
| Outcome: | The proposed framework outperforms 15 representative baselines on MBPP and MATH benchmarks. |
RouteRAG: Efficient Retrieval-Augmented Generation from Text and Graph via Reinforcement Learning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing graph-based or hybrid systems lack the ability to integrate supplementary evidence as reasoning unfolds. |
| Approach: | They propose a framework that integrates non-parametric knowledge into Large Language Models . they use a RL-based framework to optimize the entire generation process via RL . |
| Outcome: | The proposed framework outperforms existing RAG frameworks in five question answering benchmarks. |