Papers by Kelvin Guu
Controllable Semantic Parsing via Retrieval Augmentation (2021.emnlp-main)
Copied to clipboard
| Challenge: | a mechanism for enacting behavior changes without expensive model re-training would be preferable. |
| Approach: | They propose a controllable semantic parser that retrieves related exemplars from a retrieval index and augments them to the query. |
| Outcome: | The proposed model can parse queries in a new domain, adapt predictions toward specified patterns, or adapt to new semantic schemas without re-training the model. |
RARR: Researching and Revising What Language Models Say, Using Language Models (2023.acl-long)
Copied to clipboard
Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, Kelvin Guu
| Challenge: | Language models (LMs) excel at many tasks but often produce unsupported or misleading content. |
| Approach: | They propose a system that finds attribution for any text generation model and post-edits it to fix unsupported content. |
| Outcome: | The proposed system improves attribution while preserving the original output. |
Mapping natural language commands to web elements (D18-1)
Copied to clipboard
| Challenge: | a dataset of over 50,000 natural language commands captures various phenomena, including functional references, relational reasoning, and visual reasoning. |
| Approach: | They propose a task that requires the user to choose the correct element on a web page . they use a dataset of over 50,000 natural language commands to map these to web pages . |
| Outcome: | The proposed task can be viewed as a reference game based on a dataset of over 50,000 natural language commands . |
LOFT: Scalable and More Realistic Long-Context Evaluation (2025.findings-naacl)
Copied to clipboard
Jinhyuk Lee, Anthony Chen, Zhuyun Dai, Dheeru Dua, Devendra Singh Sachan, Michael Boratko, Yi Luan, Séb Arnold, Vincent Perot, Siddharth Dalmia, Hexiang Hu, Xudong Lin, Panupong Pasupat, Aida Amini, Jeremy R. Cole, Sebastian Riedel, Iftekhar Naim, Ming-Wei Chang, Kelvin Guu
| Challenge: | Long-context language models (LCLMs) can be used to perform tasks traditionally reliant on external tools like retrieval systems or databases. |
| Approach: | They propose a benchmark to evaluate LCLMs' performance on in-context retrieval and reasoning tasks using a set of tokens. |
| Outcome: | The proposed model outperforms state-of-the-art retrieval and RAG systems on in-context retrieval tasks while still requiring prompting strategies. |
Meta-Learning Fast Weight Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Dynamic evaluation of language models (LMs) adapts model parameters at test time using gradient information from previous tokens. |
| Approach: | They propose a neural component that uses gradient updates as linear attention to improve model performance. |
| Outcome: | The proposed model can be applied at training time and learn to make good use of gradient updates. |
Pretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | Recent models for unsupervised representation learning of text have put little focus on discourse-level representations. |
| Approach: | They propose an inter-sentence objective for pretraining language models that models discourse coherence and the distance between sentences. |
| Outcome: | The proposed model outperforms the BERT-Large model on the discourse representation benchmark DiscoEval and yields gains of 2%-6% absolute even for tasks that do not explicitly evaluate discourse. |
Towards Tracing Knowledge in Language Models Back to the Training Data (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work on training data attribution (TDA) may offer effective tools for identifying such examples, known as "proponents". |
| Approach: | They propose a benchmark to identify which training examples taught an LM to generate a particular factual assertion. |
| Outcome: | The proposed methods have lower proponent-retrieval precision than baselines that do not have access to the LM. |