Papers by Xinyun Chen
Symbol tuning improves in-context learning in language models (2023.emnlp-main)
Copied to clipboard
Jerry Wei, Le Hou, Andrew Lampinen, Xiangning Chen, Da Huang, Yi Tay, Xinyun Chen, Yifeng Lu, Denny Zhou, Tengyu Ma, Quoc Le
| Challenge: | Language models are sensitive to the way that prompts are given, indicating that they are not reasoning in a robust manner. |
| Approach: | They propose to fine tune language models on in-context input-label pairs where natural language labels are replaced with arbitrary symbols. |
| Outcome: | The proposed model is much stronger at reasoning tasks and more robust to underspecified prompts than the standard model. |
Exploring Underexplored Limitations of Cross-Domain Text-to-SQL Generalization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing text-to-SQL models do not generalize when faced with domain knowledge that does not frequently appear in training data. |
| Approach: | They propose a human-curated dataset based on the Spider benchmark for text-to-SQL translation. |
| Outcome: | The proposed model performs better on unseen domains than existing models on public benchmarks. |
PlotCoder: Hierarchical Decoding for Synthesizing Visualization Code in Programmatic Context (2021.acl-long)
Copied to clipboard
| Challenge: | Creating effective visualizations is a challenge given the myriad of parameters that users need to provide. |
| Approach: | They propose to synthesize visualization programs from natural language utterances and programmatic context using PlotCoder. |
| Outcome: | The proposed architecture models both the code context and the input utterance. |
Benchmarking Language Models for Code Syntax Understanding (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models capture the syntactic rules of natural languages without fine-tuning on syntax understanding tasks. |
| Approach: | They propose a benchmarking test to compare pre-trained language models with a large-scale dataset of programs annotated with syntactic relationships in their corresponding abstract syntax trees. |
| Outcome: | The proposed model fails to match baselines based on positional offsets and keywords. |
Measuring and Improving Compositional Generalization in Text-to-SQL via Component Alignment (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing models suffer performance degradation when evaluated on Spider-CG, even though every sub-sentence is seen during training. |
| Approach: | They propose a clause-level compositional example generation method to generate compositional biases from SQL clauses. |
| Outcome: | The proposed method improves generalization performance even on a training dataset. |
Exploring Label Hierarchy in a Generative Way for Hierarchical Text Classification (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for hierarchical text classification are lacking in the field of natural language processing. |
| Approach: | They propose a hierarchy-aware T5 model with path-adaptive attention mechanism to exploit hierarchical dependency across different levels. |
| Outcome: | The proposed model outperforms state-of-the-art models especially in Macro-F1 and low Macro. |
Towards Robustness of Text-to-SQL Models against Synonym Substitution (2021.acl-long)
Copied to clipboard
Yujian Gan, Xinyun Chen, Qiuping Huang, Matthew Purver, John R. Woodward, Jinxia Xie, Pengsheng Huang
| Challenge: | Existing text-to-SQL models rely on lexical matching between words in NL questions and tokens in table schemas, which may break the schema linking mechanism. |
| Approach: | They propose a human-curated dataset for text-to-SQL translation . they replace schema-related words with manually selected synonyms . |
| Outcome: | The proposed model outperforms its counterparts without the defense. |
Natural SQL: Making SQL Easier to Infer from Natural Language Specifications (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models that do not support executable SQL generation can generate executable queries. |
| Approach: | They propose an SQL intermediate representation called Natural SQL (NatSQL) they propose to preserve the core functionalities of SQL while simplifying the queries . |
| Outcome: | The proposed model outperforms existing models on a text-to-SQL benchmark . it significantly improves the performance of previous models on the same dataset . |
Re-appraising the Schema Linking for Text-to-SQL (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work has shown that schema linking reduces robustness of text-to-SQL models . EMSL is used to correlate natural language queries with the given database schema . |
| Approach: | They propose a grammar linking module to help model align grammar references with SQL keywords. |
| Outcome: | The proposed language model improves performance without using EMSL, the authors show . their language model is more robust, and the proposed grammar linking improves interoperability . |