Papers by Ian Magnusson
Extracting Fine-Grained Knowledge Graphs of Scientific Claims: Dataset and Transformer-Based Results (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches focus on high-level description of how research is carried out . instead, we focus on the subtleties of how experimental associations are presented . |
| Approach: | They propose a transformer-based approach to relational scientific information extraction that captures associations over experimental variables and their qualifications, subtypes, and evidence. |
| Outcome: | The proposed schema captures causal, comparative, predictive, statistical, and proportional associations over experimental variables along with qualifications, subtypes, and evidence. |
Exploring The Landscape of Distributional Robustness for Question Answering Models (2022.findings-emnlp)
Copied to clipboard
Anas Awadalla, Mitchell Wortsman, Gabriel Ilharco, Sewon Min, Ian Magnusson, Hannaneh Hajishirzi, Ludwig Schmidt
| Challenge: | Existing methods for predicting distributional robustness fail to generalize reliably in a variety of test conditions. |
| Approach: | They conduct a large empirical evaluation to investigate the landscape of distributional robustness in question answering. |
| Outcome: | The proposed methods are more robust to distribution shifts than fully fine-tuned models, and few-shot prompt models exhibit better robustness than few- shot prompt models. |
Scalable Data Ablation Approximations for Language Models through Modular Training and Merging (2024.emnlp-main)
Copied to clipboard
Clara Na, Ian Magnusson, Ananya Harsh Jha, Tom Sherborne, Emma Strubell, Jesse Dodge, Pradeep Dasigi
| Challenge: | Training data compositions for Large Language Models (LLMs) can significantly affect their downstream performance. |
| Approach: | They propose a method which trains individual models on subsets of a training corpus and reuses them across evaluations of combinations of subset. |
| Outcome: | The proposed method improves training efficiency by scaling only linearly with respect to new data. |
Reproducibility in NLP: What Have We Learned from the Checklist? (2023.findings-acl)
Copied to clipboard
| Challenge: | Scientific progress in NLP rests on the reproducibility of researchers’ claims. |
| Approach: | They examine 10,405 anonymous responses to the NLP Reproducibility Checklist . they find evidence of an increase in reporting of information after the Checklist's introduction . |
| Outcome: | The authors find that 44% of submissions that gather new data are 5% less likely to be accepted than those that did not. |