Papers by Dawn Lawrie
CLERC: A Dataset for U. S. Legal Case Retrieval and Retrieval-Augmented Analysis Generation (2025.findings-naacl)
Copied to clipboard
Abe Bohan Hou, Orion Weller, Guanghui Qin, Eugene Yang, Dawn Lawrie, Nils Holzenberger, Andrew Blair-Stanek, Benjamin Van Durme
| Challenge: | a dataset of case law is used to train and evaluate models for writing legal analyses . current approaches struggle to find relevant cases and generate legal analyses, authors say . |
| Approach: | They build a dataset of case law to support information retrieval and retrieval-augmented generation. |
| Outcome: | The proposed dataset supports two important backbone tasks: retrieval (IR) and retrieval-augmented generation (RAG). |
When Do Decompositions Help for Machine Reading? (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on decompositions of complex questions has focused on multi-step reasoning . but, in machine reading, it is unclear when decomposing is helpful . |
| Approach: | They conduct experiments on decompositions in machine reading to unify recent work . they find that decomposing complex questions can be helpful in zero or limited-data settings . |
| Outcome: | The proposed model can learn decompositions implicitly even with limited data, the study shows . the results are consistent with previous work on decomposing complex questions . |
Defending Against Disinformation Attacks in Open-Domain Question Answering (2024.eacl-short)
Copied to clipboard
| Challenge: | Existing methods to defend against data poisoning attacks in open-domain question answering are lacking . |
| Approach: | They propose a method that uses query augmentation to find diverse passages that could answer the original question but are less likely to have been poisoned. |
| Outcome: | The proposed method provides gains of nearly 20% exact match across varying levels of data poisoning/knowledge conflicts. |
Query Decomposition for RAG: Balancing Exploration-Exploitation (2026.eacl-long)
Copied to clipboard
Roxana Petcu, Kenton Murray, Daniel Khashabi, Evangelos Kanoulas, Maarten de Rijke, Dawn Lawrie, Kevin Duh
| Challenge: | Complex user queries often involve the exclusion of information, negation, or missing entities. |
| Approach: | They propose to decompose user requests into subqueries, retrieve potentially relevant documents for each and then aggregate them to generate an answer. |
| Outcome: | The proposed method achieves 35% gain in document-level precision and 15% increase in -nDCG . it also improves the downstream task of long-form generation. |
Pretrained Models for Multilingual Federated Learning (2022.naacl-main)
Copied to clipboard
| Challenge: | Federated Learning (FL) is a machine learning technique that trains a model across multiple distributed clients holding local data samples, without ever storing client data in a central location. |
| Approach: | They propose to use pretrained models to study three multilingual language tasks . they also examine impact of non-IID text on FL in naturally occurring data . |
| Outcome: | The proposed methods perform better than centralized learning even when using non-IID partitioning. |
Building OCR/NER Test Collections (2020.lrec-1)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is the automatic recognition of spans of text as name mentions. |
| Approach: | They propose a method for annotating named entities on transcribed text . the transcriptions are all that is needed to evaluate the performance of OCR . |
| Outcome: | The proposed collection can be used to evaluate OCR and NER on transcribed text without re-annotation . the transcriptions are all that is needed to evaluate the performance of OCR . |
NevIR: Negation in Neural Information Retrieval (2024.eacl-long)
Copied to clipboard
| Challenge: | Negation is a common everyday phenomenon and has been a consistent area of weakness for language models. |
| Approach: | They ask IR models to rank two documents that differ only by negation . they find that most current information retrieval models do not consider negation. |
| Outcome: | The proposed benchmarks show that most current models do not consider negation . the results are similar to those found in the literature, but are poorer than random ranking . |
“According to . . . ”: Prompting Language Models Improves Quoting from Pre-Training Data (2024.eacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) may hallucinate and generate false information despite pre-training on factual data. |
| Approach: | They propose a new evaluation metric that measures the extent to which model-produced answers are directly found in underlying text corpora. |
| Outcome: | The proposed evaluation metric measures the extent to which model-produced answers are directly found in underlying text corpora. |
FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions (2025.naacl-long)
Copied to clipboard
Orion Weller, Benjamin Chang, Sean MacAvaney, Kyle Lo, Arman Cohan, Benjamin Van Durme, Dawn Lawrie, Luca Soldaini
| Challenge: | Modern language models (LMs) are capable of following long and complex instructions that enable a large and diverse set of user requests. |
| Approach: | They propose a dataset that contains an instruction evaluation benchmark and a training set to help IR models learn to follow instructions. |
| Outcome: | The proposed model improves after fine-tuning on a training set and rigorous instruction evaluation benchmark. |
When do Generative Query and Document Expansions Fail? A Comprehensive Study Across Methods, Retrievers, and Datasets (2024.findings-eacl)
Copied to clipboard
| Challenge: | Using large language models (LMs) for query or document expansion can improve generalization in information retrieval. |
| Approach: | They conduct the first comprehensive analysis of large language models (LMs) for query or document expansion. |
| Outcome: | The proposed expansions improve retrieval performance for weaker models but harm stronger models. |