Rerank Before You Reason: Analyzing Reranking Tradeoffs through Effective Token Cost in Deep Search Agents (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent work emphasizes improving efficiency in LLM-based systems, especially for longcontext and multi-step reasoning. |
| Approach: | They analyze the role of listwise reranking in deep search pipelines and compare their results to a novel ETC metric to determine model scale and reasoning effort. |
| Outcome: | The proposed model scale, reasoning effort, reranking depth, and total token cost (ETC) metric improve retrieval and end-to-end accuracy and moderate reranked agents achieve comparable accuracy at substantially lower cost. |
Similar Papers
LimRank: Less is More for Reasoning-Intensive Information Reranking (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to rerank information require large-scale fine-tuning, which is computationally expensive. |
| Approach: | They propose an open-source pipeline for generating diverse, challenging, and realistic reranking examples. |
| Outcome: | The proposed model performs competitively on two benchmarks, while being trained on less than 5% of the data typically used in prior work. |
EcoRank: Budget-Constrained Text Re-ranking Using Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated dominant performance in text re-ranking. |
| Approach: | They propose a suite of budget-constrained methods to perform text re-ranking using LLMs. |
| Outcome: | The proposed method outperforms other budget-aware methods on four datasets. |
Efficiency-Effectiveness Reranking FLOPs for LLM-based Rerankers (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing studies evaluate the efficiency of LLM-based rerankers using proxy metrics such as latency and the number of forward passes. |
| Approach: | They propose to use a large language model to evaluate the efficiency of LLM-based rerankers . they propose to measure ranking quality and query processing efficiency using an interpretable FLOPs estimator . |
| Outcome: | The proposed metrics evaluate LLM-based rerankers with different architectures without running any experiments. |
FIRST: Faster Improved Listwise Reranking with Single Token Decoding (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing listwise LLMs lack efficiency as they provide ranking output in the form of a generated ordered sequence of candidate passage identifiers. |
| Approach: | They propose a listwise LLM reranking approach that leverages the first generated identifier to obtain a ranked ordering of the candidates. |
| Outcome: | The proposed approach accelerates inference by 50% while maintaining robust ranking performance with gains across BEIR benchmark. |
How Good are LLM-based Rerankers? An Empirical Analysis of State-of-the-Art Reranking Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a systematic and comprehensive empirical evaluation of state-of-the-art reranking methods is presented. |
| Approach: | They evaluate 22 reranking methods including 40 variants across established benchmarks . primary goal is to determine whether performance disparity exists between LLM-based reranters and lightweight counterparts based on novel queries . |
| Outcome: | The proposed methods perform better on familiar queries than lightweight models, the authors show . |
REARANK: Reasoning Re-ranking Agent via Reinforcement Learning (2025.emnlp-main)
Copied to clipboard
| Challenge: | REARANK is a large language model-based listwise reasoning reranking agent . it explicitly reasons be- fore reranked results, significantly improving performance and interpretability. |
| Approach: | They propose a large language model-based listwise reasoning reranking agent that explicitly reasons be- fore reranked lists. |
| Outcome: | The proposed agent outperforms GPT-4 on reasoning-intensive benchmarks and surpasses GPL-4 on BRIGHT benchmarks. |
ReasonRank: Empowering Passage Ranking with Strong Reasoning Ability (2026.acl-long)
Copied to clipboard
| Challenge: | Existing rerankers perform poorly in complex ranking scenarios due to the scarcity of reasoning-intensive training data. |
| Approach: | They propose an automated reasoning-intensive training framework which generates high-quality training labels from training queries and passages. |
| Outcome: | The proposed model outperforms baselines significantly and achieves much lower latency than the pointwise reranker. |
Are Optimal Algorithms Still Optimal? Rethinking Sorting in LLM-Based Pairwise Ranking with Batching and Caching (2025.acl-short)
Copied to clipboard
Juan Wisznia, Cecilia Bolaños, Juan Tollo, Giovanni Franco Gabriel Marraffini, Agustín Andrés Gianolini, Noe Fabian Hsueh, Luciano Del Corro
| Challenge: | a new framework for analyzing sorting algorithms in pairwise ranking prompting (PRP) is developed to re-center the cost model around LLM inferences rather than traditional pairwise comparisons. |
| Approach: | They propose a framework for analyzing sorting algorithms in pairwise ranking prompting (PRP) they propose to re-center the cost model around LLM inferences rather than traditional pairwise comparisons. |
| Outcome: | The proposed framework encourages strategies such as batching and caching to mitigate inference costs. |
Collapse of Dense Retrievers: Short, Early, and Literal Biases Outranking Factual Evidence (2025.acl-long)
Copied to clipboard
| Challenge: | Notably, when multiple biases combine, models exhibit catastrophic performance degradation, selecting the answer-containing document in less than 10% of cases over a synthetic biased document without the answer. |
| Approach: | They repurpose a relation extraction dataset to quantify the impact of heuristic biases on retrievers like Dragon+ and Contriever. |
| Outcome: | The proposed models exhibit catastrophic performance degradation when multiple biases combine, selecting the answer-containing document in less than 10% of cases over a synthetic biased document without the answer. |
ReAttn: Improving Attention-based Re-ranking via Attention Re-weighting (2026.findings-eacl)
Copied to clipboard
| Challenge: | Attention-based re-ranking methods are highly concentrated a small subset of tokens within a few documents, making others indistinguishable. |
| Approach: | They propose a post-hoc re-weighting strategy that uses attention weights to reduce lexical bias and emphasize distinctive terms. |
| Outcome: | The proposed method reduces lexical bias and emphasizes distinctive terms across documents, while maintaining a balanced distribution across informative tokens. |