Let Retrievers Think Before Action: Thought-Augmented Embedding for Dense Retrieval (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models have demonstrated that explicit step-by-step thinking can substantially improve performance on complex tasks. |
| Approach: | They propose a model that generates preliminary thoughts for input queries before document retrieval. |
| Outcome: | The proposed model generates preliminary thoughts for input queries before document retrieval. |
Similar Papers
ExpandR: Teaching Dense Retrievers Beyond Queries with LLM Guidance (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for enhancing dense retrieval with query augmentation ignore the alignment between generation and ranking objectives. |
| Approach: | They propose a unified LLM-augmented dense retrieval framework that jointly optimizes both the LLM and the retriever. |
| Outcome: | Experimental results show that ExpandR outperforms strong baselines, achieving more than 5% improvement in retrieval performance. |
Embedding-Informed Adaptive Retrieval-Augmented Generation of Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Retrieval-augmented large language models excel in various NLP tasks but are not always helpful when the knowledge required is absent in the model. |
| Approach: | They propose to determine whether the model is knowledgeable on a query via inspecting the (contextualized) pre-trained token embeddings of LLMs. |
| Outcome: | Experiments show that the proposed approach performs better than previous approaches on various benchmarks. |
Leveraging Cognitive Complexity of Texts for Contextualization in Dense Retrieval (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to estimate semantic similarity of queries and documents rely on token-level information derived from query/document interactions. |
| Approach: | They propose a new DRM that leverages query/document interactions based on full embedding representations generated by a Transformer-based model. |
| Outcome: | The proposed model outperforms fine-tuning techniques on lightweight bi-encoders and traditional late-interaction models. |
RaDeR: Reasoning-aware Dense Retrieval Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | RaDeR retrievers outperform strong baselines in reasoning tasks . large language models (LLMs) have impressive reasoning capabilities on a wide range of tasks - however, they face challenges when reasoning is needed for relevance prediction. |
| Approach: | They propose a set of reasoning-based dense retrieval models trained with data derived from mathematical problem solving using large language models. |
| Outcome: | The proposed model outperforms baselines on the BRIGHT and RAR-b benchmarks and achieves comparable or superior performance while using only 2.5% of the training data used by the concurrent work ReasonIR. |
Beyond Examples: Towards Automated Thought-level In-Context Reasoning for Large Language Models (2026.acl-long)
Copied to clipboard
Jinyang Wu, Mingkuan Feng, Shuai Zhang, Feihu Che, Zhengqi Wen, Chonghua Liao, Ling Yang, Haoran Luo, Zheng Lian, Jianhua Tao
| Challenge: | In-context learning (ICL) struggles with complex reasoning due to superficial, example-level implicit imitation. |
| Approach: | They propose an automated method that shifts from surface-level examples to more guidance-oriented thought patterns. |
| Outcome: | The proposed method achieves 80.6% accuracy on MATH and 62.5% on AMC, surpassing GPT-4o’s 77.2% and 57.5% accuracy. |
ToW: Thoughts of Words Improve Reasoning in Large Language Models (2025.naacl-long)
Copied to clipboard
Zhikun Xu, Ming Shen, Jacob Dineen, Zhaonan Li, Xiao Ye, Shijie Lu, Aswin Rrv, Chitta Baral, Ben Zhou
| Challenge: | Unlike other data augmentation methods, thoughts of words (TOW) views next-word prediction as a core reasoning task and injects fine-grained thoughts into pre-training texts. |
| Approach: | They propose a training-time data-augmentation method called thoughts of words (TOW) that injects fine-grained thoughts directly into a next-word prediction task and teaches the model to understand how the observed next word is related to previous contexts. |
| Outcome: | The proposed method reduces model hallucination by 10% and improves reasoning performance by 7% to 9% on average. |
BubbleRAG: Interactive Cognitive Offloading with Thought Bubble in Retrieval-Augmented Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Retrieval-augmented generation (RAG) extends the capabilities of large language models (LLMs) by providing access to external knowledge. |
| Approach: | They propose a framework that emulates human interactive reading through annotation and re-reading by integrating a thought bubble module that offloads internal cognition into external bookmark tokens, which are then annotated back into the context. |
| Outcome: | The proposed framework offloads internal cognition into external bookmark tokens, which are then annotated back into the context. |
Evaluating Test-Time Scaling LLMs for Legal Reasoning: OpenAI o1, DeepSeek-R1, and Beyond (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results show that Legal-R1 delivers competitive performance across diverse tasks. |
| Approach: | They propose to evaluate 12 large language models across 17 legal tasks across statutory and case-law traditions to determine their general reasoning performance. |
| Outcome: | The proposed model performs well across 17 legal tasks across statutory and case-law traditions. |
Search-o1: Agentic Search-Enhanced Large Reasoning Models (2025.emnlp-main)
Copied to clipboard
Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, Zhicheng Dou
| Challenge: | Large reasoning models (LRMs) have demonstrated impressive long stepwise reasoning capabilities through large-scale reinforcement learning. |
| Approach: | They propose a framework that enhances large reasoning models with an agentic retrieval-augmented generation mechanism and a Reason-in-Documents module for refining retrieved documents. |
| Outcome: | The proposed framework enhances LRMs with an agentic retrieval-augmented generation mechanism and Reason-in-Documents module for refining retrieved documents. |
Efficient Integration of External Knowledge to LLM-based World Models via Retrieval-Augmented Generation and Reinforcement Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing attempts to enhance LLM-based world models through prompting or fine-tuning approaches are either requiring human knowledge or computationally extensive. |
| Approach: | They propose a framework that leverages retrieval-augmented generation to integrate external knowledge to LLM-based world models. |
| Outcome: | The proposed framework outperforms baseline models and exhibits strong generalizability. |