Papers by Aaron Smith
OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokens (2025.acl-demo)
Copied to clipboard
Jiacheng Liu, Taylor Blanton, Yanai Elazar, Sewon Min, Yen-Sung Chen, Arnavi Chheda-Kothary, Huy Tran, Byron Bischoff, Eric Marsh, Michael Schmitz, Cassidy Trier, Aaron Sarnat, Jenna James, Jon Borchardt, Bailey Kuehl, Evie Yu-Yen Cheng, Karen Farley, Taira Anderson, David Albright, Carissa Schoenick, Luca Soldaini, Dirk Groeneveld, Rock Yuren Pang, Pang Wei Koh, Noah A. Smith, Sophie Lebrecht, Yejin Choi, Hannaneh Hajishirzi, Ali Farhadi, Jesse Dodge
| Challenge: | tracing language models' outputs back to training data is a problem because they are trained on text corpora with trillions of tokens . existing methods for tracers have not been scaled to work within this multi-trillion-token setting . |
| Approach: | They propose a system that traces language models' outputs verbatim back to training data . OLMOTRACE retrieves documents from the model's training data that contain exact matches . |
| Outcome: | The proposed system can find verbatim matches between LM output and training data . it can be used to explore fact checking, hallucination, and creativity of language models . |
An Investigation of the Interactions Between Pre-Trained Word Embeddings, Character Models and POS Tags in Dependency Parsing (D18-1)
Copied to clipboard
| Challenge: | Existing studies have shown that character models are less important in the presence of word embeddings, but combining them quickly leads to diminishing returns. |
| Approach: | They propose to combine pre-trained word embeddings, character models and POS tags to improve parsing quality by categorising words by frequency, POS tag and language. |
| Outcome: | The proposed system improves on initialised word embeddings but combines them quickly leads to diminishing returns. |
Parser Training with Heterogeneous Treebanks (P18-2)
Copied to clipboard
| Challenge: | In the 2017 CoNLL Shared Task on Universal Dependency Parsing, 25 languages have more than one treebank . many teams did not take advantage of the multiple treebanks, however, and trained one model per treebank instead of one model for each language. |
| Approach: | They propose a method to make the most of heterogeneous treebanks when training a monolingual parser. |
| Outcome: | The proposed method improves on training with multiple treebanks for a single language. |