Papers by Sean MacAvaney
ABNIRML: Analyzing the Behavior of Neural IR Models (2022.tacl-1)
Copied to clipboard
| Challenge: | Pretrained contextualized language models such as BERT and T5 have established a new state-of-the-art for ad-hoc ranking. |
| Approach: | They propose a framework for Analyzing the Behavior of Neural IR ModeLs that includes new types of diagnostic probes that allow us to test several characteristics that are not addressed by previous techniques. |
| Outcome: | The proposed framework tests writing styles, factuality, sensitivity to paraphrasing and word order, and can be used to identify unintended biases. |
A Deeper Look into Dependency-Based Word Embeddings (N18-4)
Copied to clipboard
| Challenge: | Word embeddings trained with dependency contexts excel at different tasks, and enhanced dependencies often improve performance. |
| Approach: | They propose to use dependency-based word embeddings to capture semantic similarity rather than relatedness. |
| Outcome: | The results show that word embeddings trained with Universal and Stanford dependencies excel at different tasks and that enhanced dependencies often improve performance. |
SLEDGE-Z: A Zero-Shot Baseline for COVID-19 Literature Search (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing search methods for COVID-19 are not based on scientific data, but use a neural re-ranking model pre-trained on scientific text. |
| Approach: | They propose a zero-shot ranking algorithm that adapts to COVID-related scientific literature . they use a neural re-ranking model pre-trained on scientific text and filters the target document . |
| Outcome: | The proposed algorithm outperforms models on the TREC COVID Round 1 leaderboard . it outperformed models that do not rely on TREC-COVID data . |
DyVo: Dynamic Vocabularies for Learned Sparse Retrieval with Entities (2024.emnlp-main)
Copied to clipboard
| Challenge: | Learned Sparse Retrieval (LSR) models use vocabularies from pre-trained transformers, which often split entities into nonsensical fragments. |
| Approach: | They propose to enhance the LSR vocabulary with Wikipedia concepts and entities, enabling the model to resolve ambiguities more effectively and stay current with evolving knowledge. |
| Outcome: | The proposed model outperforms state-of-the-art models across three entity-rich document ranking datasets. |
SMHD: a Large-Scale Resource for Exploring Online Language Usage for Multiple Mental Health Conditions (C18-1)
Copied to clipboard
| Challenge: | Existing methods to label mental health conditions are based on high-precision diagnosis patterns and carefully selected control users. |
| Approach: | They propose to use high-precision diagnosis patterns to identify self-reported diagnoses of nine different mental health conditions and obtain high-quality labeled data without manual labelling. |
| Outcome: | The proposed dataset is two orders of magnitude larger than the largest published similar resource. |
Effective Contrastive Weighting for Dense Query Expansion (2023.acl-long)
Copied to clipboard
| Challenge: | Verbatim queries that do not adequately express the user's search intent are often lexical inadequacies. |
| Approach: | They propose a contrastive weighting model that learns to select the most useful expansion embeddings for semantic search. |
| Outcome: | The proposed model outperforms existing methods while maintaining its efficiency. |
Test-time Corpus Feedback: From Retrieval to RAG (2026.findings-eacl)
Copied to clipboard
| Challenge: | Retrieval-augmented generation (RAG) pipelines treat retrieval and reasoning as isolated components, limiting performance on complex tasks. |
| Approach: | They propose to integrate large language models with retrieval to improve query quality . they also propose to use feedback to improve the query, retrieved context, or document pool . |
| Outcome: | The proposed methods bridge IR and NLP perspectives and highlight retrieval as a dynamic, learnable component of end-to-end RAG systems. |
FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions (2025.naacl-long)
Copied to clipboard
Orion Weller, Benjamin Chang, Sean MacAvaney, Kyle Lo, Arman Cohan, Benjamin Van Durme, Dawn Lawrie, Luca Soldaini
| Challenge: | Modern language models (LMs) are capable of following long and complex instructions that enable a large and diverse set of user requests. |
| Approach: | They propose a dataset that contains an instruction evaluation benchmark and a training set to help IR models learn to follow instructions. |
| Outcome: | The proposed model improves after fine-tuning on a training set and rigorous instruction evaluation benchmark. |
TBD3: A Thresholding-Based Dynamic Depression Detection from Social Media for Low-Resource Users (2022.lrec-1)
Copied to clipboard
| Challenge: | Social media are heavily used by many users to share their mental health concerns and diagnoses. |
| Approach: | They propose a dynamic thresholding technique that adjusts the classifier’s sensitivity as a function of the number of posts a user has. |
| Outcome: | The proposed method reduces the margin between users with many and few posts, on average, by 45% across all methods and increases overall performance, onaverage, by 33%. |
Exploiting Positional Bias for Query-Agnostic Generative Content in Search (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies show that neural ranking models outperform lexical models in text retrieval. |
| Approach: | They propose to exploit transformer attention mechanism to induce exploitable defects in search models through sensitivity to token position within a sequence. |
| Outcome: | The proposed model can generalise beyond a single query or topic without knowledge of topicality. |