Discovering Biases in Information Retrieval Models Using Relevance Thesaurus as Global Explanation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Currently, local explanations are not effective in predicting the model’s behavior on unseen texts. |
| Approach: | They propose a method to build a relevance thesaurus containing semantically relevant query term and document term pairs which can augment BM25 scoring functions to better approximate the neural model’s predictions. |
| Outcome: | The proposed method can augment BM25 scoring functions to better approximate the neural relevance model’s predictions. |
Similar Papers
A Counterfactual Explanation Framework for Retrieval Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing literature on explainability of information retrieval has focused on illustrating the concept of relevance concerning a retrieval model. |
| Approach: | They propose to add terms to a document to improve its ranking to answer the question of which words played a role in not being favored by a retrieval model. |
| Outcome: | The proposed framework predicts counterfactuals for statistical and deep-learning models. |
Pathway to Relevance: How Cross-Encoders Implement a Semantic Variant of BM25 (2025.emnlp-main)
Copied to clipboard
| Challenge: | Interpretability in information retrieval (IR) models is coarse-grained and poorly understood . a cross-encoder model extracts traditional relevance signals, such as term frequency and inverse document frequency . |
| Approach: | They analyze how a common IR model extracts traditional relevance signals . this is similar to the probabilistic ranking function BM25 . |
| Outcome: | The proposed model extracts traditional relevance signals in early-to-middle layers, similar to BM25 . the model then combine these concepts in later layers, laying the groundwork for future interventions . |
Learning to Describe Unknown Phrases with Local and Global Contexts (N19-1)
Copied to clipboard
Shonosuke Ishiwatari, Hiroaki Hayashi, Naoki Yoshinaga, Graham Neubig, Shoetsu Sato, Masashi Toyoda, Masaru Kitsuregawa
| Challenge: | Existing methods for contextual guessing and definition generation do not take clues from local contexts. |
| Approach: | They propose a neural description model that takes clues from local and global contexts . they assume that the target phrase is newly emerged and there is no global context . |
| Outcome: | The proposed model takes clues from local and global contexts over existing methods . it is more effective than existing methods for non-standard English explanation . |
It’s All Relative: Learning Interpretable Models for Scoring Subjective Bias in Documents from Pairwise Comparisons (2024.eacl-long)
Copied to clipboard
| Challenge: | a new model to score subjective bias in documents is developed to perform pairwise comparisons . a recent study shows that the model can be explained and validated for other domains based on the training data. |
| Approach: | They propose an interpretable model to score subjective bias in Wikipedia articles . they train the model on pairs of revisions of the same Wikipedia article . |
| Outcome: | The proposed model can interpret parameters to discover words most indicative of bias . it compares legal texts, news media and law amendments in three settings . |
Contextual Relevance and Adaptive Sampling for LLM-Based Document Reranking (2026.acl-long)
Copied to clipboard
| Challenge: | identifying relevant documents for Reasoning-intensive queries remains a challenge . large language models have shown strong performance in zero-shot document reranking . |
| Approach: | They propose a reranking algorithm that estimates contextual relevance by aggregating LLMs' relevance judgments across batches. |
| Outcome: | The proposed algorithm improves nDCG@10 over retrieval and reranking baselines by 15% and 6–21% respectively. |
Inflating Topic Relevance with Ideology: A Case Study of Political Ideology Bias in Social Topic Detection Models (2020.coling-main)
Copied to clipboard
| Challenge: | a study examines the impact of political ideology biases in training data . topic detection methods may contain or propagate certain biase resulting in a skewed data collection . |
| Approach: | They propose to learn a text representation that is invariant to political ideology while still judging topic relevance. |
| Outcome: | The proposed model can be invariant to political ideology while still judging topic relevance. |
Cross-Lingual Training of Neural Models for Document Ranking (2020.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that multi-lingual BERT models can be used for document ranking in non-English languages . a blog post by Google suggests that the company is exploring this approach to improve web search across a number of languages. |
| Approach: | They propose to leverage relevance judgments in English to train neural document ranking models for mono-lingual retrieval in multiple target languages. |
| Outcome: | The proposed approach improves search quality in non-English languages while requiring low resources. |
Contextualized Word Representations for Reading Comprehension (N18-2)
Copied to clipboard
| Challenge: | Reading comprehension (RC) is a high-level task in natural language understanding that requires reading a document and answering questions about its content. |
| Approach: | They propose to provide a standard neural network for reading a document and answering a question about its content. |
| Outcome: | The proposed model improves on the competitive SQuAD dataset by providing rich contextualized word representations and allowing it to choose between context-dependent and context-independent representations. |
Explain then Rank: Scale Calibration of Neural Rankers Using Natural Language Explanations from LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | Neural ranking models produce the final document scores, but they are often treated as transient information and only the relative orderings are preserved to produce a ranking. |
| Approach: | They propose to exploit large language models (LLMs) to provide relevance and uncertainty signals for these neural text rankers to produce scale-calibrated scores through Monte Carlo sampling of natural language explanations (NLEs). |
| Outcome: | The proposed approach outperforms previous calibration methods and LLM-based methods for ranking, calibration, and query performance prediction tasks. |
Incorporating Relevance Feedback for Information-Seeking Retrieval using Few-Shot Document Re-Ranking (2022.emnlp-main)
Copied to clipboard
| Challenge: | Extensive experiments demonstrate that integrating relevance feedback directly into neural re-ranking models improves their performance. |
| Approach: | They propose a kNN approach that re-ranks documents based on their similarity with the query and the documents the user considers relevant. |
| Outcome: | The proposed approach outperforms all other methods by 5.2% on large-scale information retrieval datasets. |