Papers by Aditi Raghunathan
Not-Just-Scaling Laws: Towards a Better Understanding of the Downstream Impact of Language Model Design Decisions (2025.emnlp-main)
Copied to clipboard
Emmy Liu, Amanda Bertsch, Lintang Sutawika, Lindia Tjuatja, Patrick Fernandes, Lara Marinov, Michael Chen, Shreya Singhal, Carolin Lawrence, Aditi Raghunathan, Kiril Gashteovski, Graham Neubig
| Challenge: | Language model performance is largely dependent on pretraining decisions, but scaling laws based on only these two aspects do not always explain downstream task performance. |
| Approach: | They meta-analyze 92 open-source pretrained models to quantify their impact on performance. |
| Outcome: | The framework lays a foundation for more systematic investigation of how model development choices shape final capabilities. |
Mitigating Bias in RAG: Controlling the Embedder (2025.findings-acl)
Copied to clipboard
| Challenge: | a promising modular AI system enhances factuality and privacy in large language models . however, each component introduces its own biases into the RAG system, which could cause representational harm and unsafe user interactions. |
| Approach: | They study the conflict between biases of each component and their relationship to the overall bias of the retrieval augmented generation system. |
| Outcome: | The proposed model can be controlled by the embedder while maintaining utility . the proposed model is more fair than existing models, the authors show . |
Understanding the Influence of Synthetic Data for Text Embedders (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in general purpose text embedders have been driven by training on synthetic training data. |
| Approach: | They propose to use GPT-4 to produce high quality synthetic data that expands existing training datasets for embeddings to new tasks. |
| Outcome: | The proposed dataset is high quality and leads to consistent improvements in performance. |
On the Feasibility of In-Context Probing for Data Attribution (2025.findings-naacl)
Copied to clipboard
| Challenge: | In-context probing (ICP) can be used to identify training data that contributes to model outputs, but many data attribution methods, such as influence functions, use model gradients and are computationally expensive. |
| Approach: | They propose to use in-context probing (ICP) to proxy for gradient-based data attribution for data selection under conditions contingent on data similarity. |
| Outcome: | The proposed method can be used to identify training data that contribute to model outputs and fine tune models on training data. |
Robust Encodings: A Framework for Combating Adversarial Typos (2020.acl-main)
Copied to clipboard
| Challenge: | Existing procedures to defend against adversarial perturbations of inputs are heuristic in nature and susceptible to stronger attacks. |
| Approach: | They propose a framework that provides robustness without compromises on model architecture. |
| Outcome: | The proposed framework achieves robustness on six tasks while incompatible with state-of-the-art models like BERT. |
Certified Robustness to Adversarial Word Substitutions (D19-1)
Copied to clipboard
| Challenge: | State-of-the-art NLP models can be fooled by adversaries that apply seemingly innocuous label-preserving transformations to input text. |
| Approach: | They propose to train models that are provably robust to all word substitutions in a family of label-preserving transformations that can be replaced with a similar word without changing the original sentiment. |
| Outcome: | The proposed models achieve 75% adversarial accuracy on both sentiment analysis and natural language inference on IMDB and SNLI compared to models trained normally and ones trained with data augmentation. |