Papers by Zihuiwen Ye
Augmenting Multi-Turn Text-to-SQL Datasets with Self-Play (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Numerous architectures and pretraining methods have been proposed for context-dependent text-to-SQL, but the size of the datasets used has been limited due to the high cost of annotating multi-turn dialogue and SQL pairs. |
| Approach: | They propose to augment training datasets using self-play which leverages contextual information to synthesize new interactions to adapt the model to new databases. |
| Outcome: | The proposed model improves accuracy on SParC and CoSQL, two widely used cross-domain text-to-SQl datasets. |
ExplainaBoard: An Explainable Leaderboard for NLP (2021.acl-demo)
Copied to clipboard
Pengfei Liu, Jinlan Fu, Yang Xiao, Weizhe Yuan, Shuaichen Chang, Junqi Dai, Yixin Liu, Zihuiwen Ye, Graham Neubig
| Challenge: | Using leaderboards, researchers can track the performance of various systems on various NLP tasks. |
| Approach: | They propose a new conceptualization and implementation of NLP evaluation using a leaderboard. |
| Outcome: | The ExplainaBoard is an evaluation tool for natural language processing (NLP) it covers more than 400 systems, 50 datasets, 40 languages, and 12 tasks. |
Towards More Fine-grained and Reliable NLP Performance Prediction (2021.eacl-main)
Copied to clipboard
| Challenge: | Performance prediction is a task of estimating a system’s performance without performing experiments. |
| Approach: | They propose to understand reliability of performance prediction models from two angles: confidence intervals and calibration. |
| Outcome: | The proposed methods demonstrate the feasibility of fine-grained performance prediction and the necessity to perform reliability analysis for performance prediction methods in the future. |
Improving Reward Models with Synthetic Critiques (2025.findings-naacl)
Copied to clipboard
| Challenge: | a recent study shows that reward models overfit on superficial features, hindering generalization performance . prevailing approach to training preference-based reward models presents several challenges . |
| Approach: | They propose a method that uses synthetic natural language critiques to provide additional feedback to large language models. |
| Outcome: | The proposed approach improves performance and data efficiency of RMs initialized from different pretrained models, reducing the reliance on costly human annotations. |