Papers by Daniela Gottesman
Hopping Too Late: Exploring the Limitations of Large Language Models on Multi-Hop Queries (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) can solve complex multi-step problems, but little is known about how these computations are implemented internally. |
| Approach: | They propose a "back-patching" analysis method to solve multi-hop queries . they propose resolving the bridge entity into the bridge and the second hop into the target entity into latent steps. |
| Outcome: | The proposed method solves multi-hop queries that require two information extraction steps . it shows that the later layers lack the necessary knowledge to correctly generate the answer . |
Performance Gap in Entity Knowledge Extraction Across Modalities in Vision Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Vision-language models excel at extracting and reasoning about information from images, yet their capacity to leverage internal knowledge about specific entities remains underexplored. |
| Approach: | They propose a dataset which allows separating entity recognition and question answering . they hypothesize that this decline arises from limitations in how information flows from image tokens to query tokens. |
| Outcome: | The proposed model performance drops when the entity is presented visually rather than textually. |
Eliciting Textual Descriptions from Representations of Continuous Prompts (2025.findings-acl)
Copied to clipboard
| Challenge: | Prior attempts to interpret continuous prompts relied on projecting individual tokens onto the vocabulary space, but this approach yielded arbitrary or contradictory text. |
| Approach: | They propose to project individual prompt tokens onto the model's vocabulary space and interpret each prompt token individually. |
| Outcome: | The proposed method yields accurate task descriptions and is more faithful as task performance increases. |
How Well Can Reasoning Models Identify and Recover from Unhelpful Thoughts? (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent reasoning models show the ability to reflect, backtrack, and self-validate their reasoning . |
| Approach: | They investigate how well models identify and recover from unhelpful thoughts . they show that models are effective at identifying most unhelpfully thoughts - but struggle to recover from the same thoughts when they are injected into their thinking process . |
| Outcome: | The proposed model can identify unhelpful thoughts but struggle to recover from them . larger models struggle more than smaller models to recover short irrelevant thoughts . |
Estimating Knowledge in Large Language Models Without Generating a Single Token (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to evaluate knowledge in large language models require querying and evaluating the model's generated responses. |
| Approach: | They ask whether it is possible to estimate how knowledgeable a model is about a subject entity only from its internal computation. |
| Outcome: | The proposed model performs well with QA accuracy and FActScore . it can be leveraged to guide decisions on how to apply further training or augment queries with retrieval. |