Papers by Emma Strubell
Energy and Carbon Considerations of Fine-Tuning BERT (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work quantifying energy costs and associated carbon emissions has focused on pretraining and fine-tuning. |
| Approach: | They perform an empirical study to quantify the energy requirements of language model fine-tuning in the context of pretraining and inference. |
| Outcome: | The proposed model fine-tuning energy and carbon footprints are compared with pre-training and inference energy requirements and outline recommendations for NLP researchers and practitioners. |
To Adapt or to Annotate: Challenges and Interventions for Domain Adaptation in Open-Domain Question Answering (2023.acl-long)
Copied to clipboard
| Challenge: | Recent advances in open-domain question answering have demonstrated impressive accuracy on general-purpose domains like Wikipedia. |
| Approach: | They propose a more realistic end-to-end domain shift evaluation setting covering five diverse domains to assess model adaption. |
| Outcome: | The proposed model improves by 24 points when adapted to unsupervised datasets. |
Transfer Learning from Semantic Role Labeling to Event Argument Extraction with Template-based Slot Querying (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for event argument extraction are limited by the limited amount of annotations available. |
| Approach: | They propose to use SRL annotations for event argument extraction . they propose to specify natural language-like queries to tackle label mismatch problem . |
| Outcome: | The proposed model achieves impressive zero-shot results on English benchmarks . it also provides benefits in low-resource cases, where few annotations are available . |
Beyond Text: Characterizing Domain Expert Needs in Document Research (2025.findings-acl)
Copied to clipboard
| Challenge: | Document research is a key part of almost all knowledge work, but are text-based NLP systems able to model these tasks as experts conceptualize and perform them? |
| Approach: | They interview 16 domain experts to understand their processes of document research . they find that processes are idiosyncratic, iterative, and rely heavily on social context . |
| Outcome: | The findings show that document research processes are idiosyncratic, iterative, and rely heavily on the social context of a document in addition to its content. |
Stereotype or Personalization? User Identity Biases Chatbot Recommendations (2025.findings-acl)
Copied to clipboard
| Challenge: | We show that when people use large language models to generate recommendations, the LLMs produce responses that reflect both what the user wants and who the user is. |
| Approach: | They propose that chatbots should transparently indicate when user’s revealed identity influences model recommendations but fail to do so . |
| Outcome: | The proposed model generates racially stereotypical recommendations regardless of whether the user revealed their identity intentionally or unintentionally through implicit cues. |
Train Flat, Then Compress: Sharpness-Aware Minimization Learns More Compressible Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in hardware, modeling, and optimization for deep neural networks have led to improvements in memory and inference efficiency. |
| Approach: | They propose to combine sharpness-aware minimization with various model compression methods to improve model compressibility. |
| Outcome: | Empirically, optimizing for flatter minima leads to greater compressibility of parameters compared to vanilla Adam when fine-tuning BERT models, with little to no loss in accuracy on the GLUE text classification and SQuAD question answering benchmarks. |
To Build Our Future, We Must Know Our Past: Contextualizing Paradigm Shifts in Natural Language Processing (2023.emnlp-main)
Copied to clipboard
| Challenge: | Natural language processing (NLP) is in a period of disruptive change that is impacting our methodologies, funding sources, and public perception. |
| Approach: | They conduct interviews with 26 NLP researchers of varying seniority, research area, institution, and social identity to identify cyclical patterns in the field and new shifts without historical parallel . they conclude by discussing shared visions, concerns, and hopes for the future of NLP . |
| Outcome: | The authors identify cyclical patterns in the field, as well as new shifts without historical parallel, including changes in benchmark culture and software infrastructure. |
DSI++: Updating Transformer Memory with New Documents (2023.emnlp-main)
Copied to clipboard
Sanket Mehta, Jai Gupta, Yi Tay, Mostafa Dehghani, Vinh Tran, Jinfeng Rao, Marc Najork, Emma Strubell, Donald Metzler
| Challenge: | Differentiable Search Indices (DSIs) encode a corpus of documents and use the same model to map queries directly to relevant document identifiers. |
| Approach: | They propose a continual learning challenge for Differentiable Search Indices (DSIs) they propose to continuously index new documents while answering queries related to previously and newly indexed documents. |
| Outcome: | The proposed model stably memorizes more documents and improves the average Hits@10 by +21.1% over baselines. |
The Framework Tax: Disparities Between Inference Efficiency in NLP Research and Deployment (2023.emnlp-main)
Copied to clipboard
| Challenge: | Inference is estimated to make up 80 to 90% of ML cloud computing demand . |
| Approach: | They propose to identify bottlenecks in deep learning frameworks that are causing the disparity in model latency as hardware speed increases over time. |
| Outcome: | The proposed models show that the framework tax is increasing as the hardware speed increases over time. |
Understanding the Effect of Model Compression on Social Bias in Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) trained with self-supervision on vast corpora of web text fit to the social biases of that text, leading to representational harm. |
| Approach: | They propose to use quantization and knowledge distillation to reduce the computational burden of LLMs to mitigate the effects of inappropriate social biases learned during pretraining. |
| Outcome: | The proposed methods reduce the computational burden of large language models by reducing their size and complexity. |
Efficient Methods for Natural Language Processing: A Survey (2023.tacl-1)
Copied to clipboard
Marcos Treviso, Ji-Ung Lee, Tianchu Ji, Betty van Aken, Qingqing Cao, Manuel R. Ciosici, Michael Hassid, Kenneth Heafield, Sara Hooker, Colin Raffel, Pedro H. Martins, André F. T. Martins, Jessica Zosa Forde, Peter Milder, Edwin Simpson, Noam Slonim, Jesse Dodge, Emma Strubell, Niranjan Balasubramanian, Leon Derczynski, Iryna Gurevych, Roy Schwartz
| Challenge: | Recent work in natural language processing (NLP) has yielded appealing results from scaling model parameters and training data, but using only scale to improve performance means resource consumption also grows. |
| Approach: | They propose to use data, time, storage, or energy to improve model performance. |
| Outcome: | The proposed methods and findings provide guidance for conducting NLP under limited resources and point towards promising research directions for developing more efficient methods. |
Decomposing Unitization and Typing for Efficient and Consistent Span-Bound Concept Annotation (2026.findings-acl)
Copied to clipboard
| Challenge: | Substantial resources are typically spent on unitizing, the task of identifying precise span boundaries for entity mentions. |
| Approach: | They propose a method that focuses manual efforts on typed position annotations instead of full concept annotation. |
| Outcome: | The proposed procedure reduces the cost of concept annotations by focusing on typed positions instead of full concept annotation. |
Bridging Fairness and Environmental Sustainability in Natural Language Processing (2022.emnlp-main)
Copied to clipboard
| Challenge: | a lack of research on the interplay between fairness and environmental impact is a problem in natural language processing . fairness is prone to encode and amplify stereotypical social biases, according to several studies . |
| Approach: | They evaluate a technique to reduce energy consumption of English NLP models by knowledge distillation for its impact on fairness. |
| Outcome: | The proposed method reduces energy consumption and environmental impact of English NLP models. |
On the Benefit of Syntactic Supervision for Cross-lingual Transfer in Semantic Role Labeling (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in neural architectures and pre-trained representations have greatly improved the performance of fully-supervised semantic role labeling (SRL) but there are limitations in the availability of supervised training data. |
| Approach: | They propose to leverage syntactic dependencies to facilitate cross-lingual transfer by annotating predicate-argument structures in text. |
| Outcome: | The proposed model can be extended to other languages with limited training data. |
Data-efficient Active Learning for Structured Prediction with Partial Annotation and Self-Training (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Structured prediction is a fundamental problem in NLP, wherein the label space consists of complex structured outputs with groups of interdependent variables. |
| Approach: | They propose a partial annotation approach that selects only the most informative sub-structures for annotation and a method that incorporates the current model's automatic predictions as pseudo-labels for un-annotated sub-structurals. |
| Outcome: | The proposed approach reduces annotation cost over strong full annotation baselines under a fair comparison scheme that takes reading time into consideration. |
Simultaneously Self-Attending to All Mentions for Full-Abstract Biological Relation Extraction (N18-1)
Copied to clipboard
| Challenge: | Existing work in relation extraction ignores relationships expressed across sentence boundaries . document-level annotation is common in biological text . |
| Approach: | They propose a model which simultaneously predicts relationships between all mention pairs in a document. |
| Outcome: | The proposed model is larger than existing human-annotated biological information extraction datasets and more accurate than distantly supervised alternatives. |
Scalable Data Ablation Approximations for Language Models through Modular Training and Merging (2024.emnlp-main)
Copied to clipboard
Clara Na, Ian Magnusson, Ananya Harsh Jha, Tom Sherborne, Emma Strubell, Jesse Dodge, Pradeep Dasigi
| Challenge: | Training data compositions for Large Language Models (LLMs) can significantly affect their downstream performance. |
| Approach: | They propose a method which trains individual models on subsets of a training corpus and reuses them across evaluations of combinations of subset. |
| Outcome: | The proposed method improves training efficiency by scaling only linearly with respect to new data. |
Gradient Localization Improves Lifelong Pretraining of Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for continual learning do not account for locality of knowledge . however, in practice language models are deployed in dynamic real-world settings and their learned knowledge becomes stale over time. |
| Approach: | They examine two types of knowledge relating to temporally sensitive entities . they hypothesize that lack of consideration of locality contributes to failed uptake of new information . |
| Outcome: | The proposed model can be improved by updating parameters to relevant layers . the proposed model is based on a large static web-scale dataset . |
Energy and Policy Considerations for Deep Learning in NLP (P19-1)
Copied to clipboard
| Challenge: | Recent advances in hardware and methodology for training neural networks have enabled significant accuracy improvements across many NLP tasks. |
| Approach: | They quantify the approximate financial and environmental costs of training neural network models . they propose actionable recommendations to reduce costs and improve equity in NLP research . |
| Outcome: | The proposed recommendations address the cost and environmental costs of training neural networks for NLP. |
FicSim: A Dataset for Multi-Faceted Semantic Similarity in Long-Form Fiction (2025.findings-emnlp)
Copied to clipboard
| Challenge: | evaluating the usefulness of language models for literary-domain tasks remains challenging due to the cost of fine-grained annotation for long-form texts and data contamination concerns inherent in using public-domain literature. |
| Approach: | They use a dataset of long-form, recently written fiction to evaluate embedding models . they prioritize author agency and rely on continual, informed author consent . |
| Outcome: | The proposed dataset of long-form, recently written fiction is compared with existing models on this task. |
Energy Considerations of Large Language Model Inference and Efficiency Optimizations (2025.acl-long)
Copied to clipboard
| Challenge: | Prior benchmarking efforts focused on latency reduction in idealized settings, often overlooking real-world inference workloads that shape energy use. |
| Approach: | They propose a modeling approach that approximates real-world LLM workflows . they show that the effectiveness of inference optimizations is sensitive to workload geometry . |
| Outcome: | The proposed approach reduces energy use by 73% from unoptimized baselines. |
Annotating Mentions Alone Enables Efficient Domain Adaptation for Coreference Resolution (2023.acl-long)
Copied to clipboard
| Challenge: | Recent results show that annotating mentions is twice as fast as annotation of full coreference chains. |
| Approach: | They propose a method for efficiently adapting coreference models using only mentions in the target domain without increasing annotator time. |
| Outcome: | The proposed method improves average F1 without increasing annotator time. |
AboutMe: Using Self-Descriptions in Webpages to Document the Effects of English Pretraining Data Filters (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models' (LLMs) abilities are drawn from their pretraining data. however, decisions around what data is retained or removed during this initial stage are under-scrutinized. |
| Approach: | They ground web text, a popular pretraining data source, to its social and geographic contexts. |
| Outcome: | The results show that some quality classifiers act like topical domain filters, and langID overlook English content from some regions of the world. |
Linguistically-Informed Self-Attention for Semantic Role Labeling (D18-1)
Copied to clipboard
| Challenge: | Existing models of semantic role labeling use no explicit linguistic features. prior work has shown that syntax trees can dramatically improve SRL decoding. |
| Approach: | They propose a neural network model that incorporates syntax using only raw tokens . they show that LISA out-performs the state-of-the-art with contextually-encoded word representations a 1.0 F1 on newswire and 2.0 F1 in out-of domain text . |
| Outcome: | The proposed model outperforms the state-of-the-art model with word embeddings and predicted predicates. |
A Survey of Active Learning for Natural Language Processing (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing literature surveys on active learning for NLP are too specific or too general, covering deep active learning. |
| Approach: | They propose to use active learning to improve model learning and annotation cost for NLP problems. |
| Outcome: | The proposed approach is based on a large dataset of data-driven machine learning models. |
Improving Compositional Generalization with Self-Training for Data-to-Text Generation (2022.acl-long)
Copied to clipboard
| Challenge: | Data-to-text generation focuses on generating fluent natural language responses from structured meaning representations (MRs). |
| Approach: | They propose a template-based input representation that greatly improves the model’s generalization capability. |
| Outcome: | The proposed model improves tree accuracy by 46%+ and reduces slot error rates by 73%+ over the strong baselines on SGD and Weather benchmarks. |