Papers by Zora Zhiruo Wang
CodeRAG-Bench: Can Retrieval Augment Code Generation? (2025.findings-naacl)
Copied to clipboard
Zora Zhiruo Wang, Akari Asai, Xinyan Velocity Yu, Frank F. Xu, Yiqing Xie, Graham Neubig, Daniel Fried
| Challenge: | Language models excel at generating code, but many programs are difficult to generate using only parametric knowledge. |
| Approach: | They propose a retrieval-augmented code generation benchmark that provides reproducible evaluations on retrieval and end-to-end code generation performance. |
| Outcome: | The proposed benchmark covers programming, open-domain, and repository-level tasks and provides reproducible evaluations on retrieval and end-to-end code generation performance. |
Benchmarking Failures in Tool-Augmented Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | FAIL-TaLMs contains 1,749 examples using 906 tools across 21 categories, including single- and multi-tool usage. |
| Approach: | They introduce a benchmark to examine the shortcomings of tool-augmented language models (TaLMs) that assume 'perfect' information access and tool availability. |
| Outcome: | The proposed benchmark systematically evaluates 1,749 examples using 906 tools across 21 categories, including single- and multi-tool usage. |
cAST: Enhancing Code Retrieval-Augmented Generation with Structural Chunking via Abstract Syntax Tree (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing line-based chunking heuristics often break semantic structures, splitting functions or merging unrelated code. |
| Approach: | They propose a structure-aware method that breaks large AST nodes into smaller chunks . this method generates self-contained, semantically coherent units across programming languages . |
| Outcome: | The proposed method boosts Recall@5 by 4.3 points on RepoEval retrieval and Pass@1 by 2.67 points on SWE-bench generation. |