Papers by Zhiruo Wang
ECCO: Can We Improve Model-Generated Code Efficiency Without Sacrificing Functional Correctness? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Current methods for optimizing program efficiency improve performance measured by execution time, but they often come at the cost of severely decreasing the functional correctness. |
| Approach: | They propose a reproducible benchmark for evaluating program efficiency via two paradigms: natural language (NL) based code generation and history-based code editing. |
| Outcome: | The proposed approach improves performance while maintaining correctness while adding execution information. |
Execution-Based Evaluation for Open-Domain Code Generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | ODEX is the first open-domain EXecution-based natural language (NL) to Python code generation dataset. |
| Approach: | They propose to use a dataset to extend the scope of coding queries to more realistic settings by using open-domain EXecution-based natural language (NL) to Python. |
| Outcome: | The proposed dataset has 945 NL-Code pairs and 1,707 human-written test cases. |
API-Assisted Code Generation for Question Answering on Varied Table Structures (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to table question answering have been limited to specific table structures. |
| Approach: | They propose a unified TableQA framework that uses Python as a querying language and few-shot prompting to translate NL questions into Python programs. |
| Outcome: | The proposed framework provides a unified representation for structured tables as multi-index Pandas data frames and uses Python as a powerful querying language to translate NL questions into Python programs. |
HiTab: A Hierarchical Table Dataset for Question Answering and Natural Language Generation (2022.acl-long)
Copied to clipboard
Zhoujun Cheng, Haoyu Dong, Zhiruo Wang, Ran Jia, Jiaqi Guo, Yan Gao, Shi Han, Jian-Guang Lou, Dongmei Zhang
| Challenge: | Existing studies on table reasoning focus on flat tables and hierarchical tables . a new dataset, HiTab, aims to examine numerical reasoning over hierarchic tables based on hierarchically structured tables - a strong challenge for existing baselines and a valuable benchmark for future research. |
| Approach: | They propose a hierarchical question answering and natural language generation dataset to study hierarchic tables. |
| Outcome: | The proposed model shows that it is effective in QA and natural language generation over hierarchical tables. |
Retrieval as Attention: End-to-end Learning of Retrieval and Reading within a Single Transformer (2022.emnlp-main)
Copied to clipboard
| Challenge: | eschewing separate architecture and training for knowledge-intensive tasks is cumbersome . end-to-end training only based on supervision from the end task is awkward . |
| Approach: | They propose a single Transformer that performs retrieval as attention and end-to-end training solely based on supervision from the end QA task. |
| Outcome: | The proposed model outperforms state-of-the-art retrievers and readers on in-domain datasets. |
FastBERT: a Self-distilling BERT with Adaptive Inference Time (2020.acl-main)
Copied to clipboard
| Challenge: | Pre-trained language models like BERT have proven to be highly performant, but are often computationally expensive in many practical scenarios. |
| Approach: | They propose a speed-tunable FastBERT with adaptive inference time that can be flexibly adjusted under varying demands. |
| Outcome: | The proposed model achieves promising results in English and Chinese datasets. |
MCoNaLa: A Benchmark for Code Generation from Multiple Natural Languages (2023.findings-eacl)
Copied to clipboard
| Challenge: | MCoNaLa benchmarks natural language code generation in languages that are not native to English. |
| Approach: | They propose to benchmark natural language code generation from natural language commands extending beyond English by using a multilingual dataset. |
| Outcome: | The proposed dataset compares natural language commands with code generation systems in three languages. |