Papers with CODEX
On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codex (2023.eacl-main)
Copied to clipboard
Terry Yue Zhuo, Zhuang Li, Yujin Huang, Fatemeh Shiri, Weiqing Wang, Gholamreza Haffari, Yuan-Fang Li
| Challenge: | Existing techniques for parsing natural-language utterances are vulnerable to adversarial attacks, requiring large amounts of labelled data and expensive human annotation. |
| Approach: | They propose to enhance the adversarial robustness of a prompt-based semantic parser based on a language model trained on code by constructing a set of demonstration examples. |
| Outcome: | The proposed method can be enhanced without significant amounts of labelled data or expensive human annotations on in-domain semantic parsing data. |
Execution-Based Evaluation for Open-Domain Code Generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | ODEX is the first open-domain EXecution-based natural language (NL) to Python code generation dataset. |
| Approach: | They propose to use a dataset to extend the scope of coding queries to more realistic settings by using open-domain EXecution-based natural language (NL) to Python. |
| Outcome: | The proposed dataset has 945 NL-Code pairs and 1,707 human-written test cases. |