Papers by Alex Gu
The Counterfeit Conundrum: Can Code Language Models Grasp the Nuances of Their Incorrect Generations? (2024.findings-acl)
Copied to clipboard
| Challenge: | Language models are more proficient at code generation, but they still generate incorrect programs. |
| Approach: | They define a group of models that have a high log-probability and weak correctness checks. |
| Outcome: | The proposed model samples fail to understand counterfeits through three clear failure modes . counterfeits are confusing to the model as they are to other models, the authors say . |
LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Provers (2023.emnlp-main)
Copied to clipboard
Theo Olausson, Alex Gu, Ben Lipkin, Cedegao Zhang, Armando Solar-Lezama, Joshua Tenenbaum, Roger Levy
| Challenge: | Logical reasoning is an important task for artificial intelligence, says a new study . many prompting-based strategies to enable large language models fail in subtle and unpredictable ways. |
| Approach: | They propose to reformulate logical reasoning tasks by leveraging large language models . they use a modular neurosymbolic programming approach to translate premises and conclusions from natural language to logic . |
| Outcome: | The proposed approach outperforms open-source models on FOLIO and ProofWriter while showing distinct failure modes. |
Language Agnostic Code Embeddings (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent studies show that code language models have strong cross-lingual traits, but their multilingual representations can be dissected into a language-specific syntax component and a semantic component. |
| Approach: | They propose to isolate and eliminate language-specific components from multilingual code embeddings to improve downstream code retrieval tasks. |
| Outcome: | The proposed model improves retrieval tasks by removing language-specific components . the proposed model can be used to perform a variety of code generation tasks . |
Few-shot In-context Learning on Knowledge Base Question Answering (2023.acl-long)
Copied to clipboard
| Challenge: | KB-BINDER enables few-shot in-context learning over knowledge base questions . KBQA is a difficult problem due to the heterogeneity of knowledge bases . |
| Approach: | They propose a framework that enables few-shot in-context learning over KBQA tasks. |
| Outcome: | The proposed framework can outperform state-of-the-art models on GraphQA and MetaQA datasets. |
Scaling Collaborative Effort with Agents (2026.findings-acl)
Copied to clipboard
Shannon Zejiang Shen, Valerie Chen, Ken Gu, Alexis Ross, Zixian Ma, Jillian Ross, Alex Gu, Chenglei Si, Wayne Chi, Andi Peng, Jocelyn J Shen, Ameet Talwalkar, Tongshuang Wu, David Sontag
| Challenge: | Current evaluations of agents focus on producing high-quality, final outputs in one shot, failing to account for the inherently iterative nature of many real-world problems. |
| Approach: | They propose a framework that captures how an agent’s utility grows with increasing user involvement. |
| Outcome: | The proposed framework captures how an agent’s utility grows with increasing user involvement, revealing a missing ingredient in agent design: the ability to sustain engagement and scaffold user understanding. |