Papers by Alvin Cheung
PlotCoder: Hierarchical Decoding for Synthesizing Visualization Code in Programmatic Context (2021.acl-long)
Copied to clipboard
| Challenge: | Creating effective visualizations is a challenge given the myriad of parameters that users need to provide. |
| Approach: | They propose to synthesize visualization programs from natural language utterances and programmatic context using PlotCoder. |
| Outcome: | The proposed architecture models both the code context and the input utterance. |
SlimFit: Memory-Efficient Fine-Tuning of Transformer-based Models Using Training Dynamics (2024.naacl-long)
Copied to clipboard
Arash Ardakani, Altan Haan, Shangyin Tan, Doru Thom Popovici, Alvin Cheung, Costin Iancu, Koushik Sen
| Challenge: | SlimFit reduces the memory requirements of transformer-based models by analyzing their training dynamics and freezing less-contributory layers during fine-tuning. |
| Approach: | They propose a tool that analyzes transformer-based models and freezes less-contributory layers during fine-tuning to reduce the overall on-device memory usage. |
| Outcome: | SlimFit reduces the memory requirements of transformer-based models by analyzing their training dynamics and freezing less-contributory layers during fine-tuning. |
Model-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers (2023.acl-long)
Copied to clipboard
Linyuan Gong, Chenyan Xiong, Xiaodong Liu, Payal Bajaj, Yiqing Xie, Alvin Cheung, Jianfeng Gao, Xia Song
| Challenge: | Recent work in NLP has shown that pretrained language models have made notable progress toward generalization to unseen tasks. |
| Approach: | They propose to pretrain T5 using an auxiliary model to construct more challenging token replacements for the main model to denoise. |
| Outcome: | The proposed model outperforms similar-sized baseline models on prompted NLP benchmarks and rivals the state-of-the-art model with only **8%** of its parameters. |
Learning Programmatic Idioms for Scalable Semantic Parsing (D19-1)
Copied to clipboard
| Challenge: | In state-of-the-art semantic parsers map natural language instructions to source code . idioms improve the accuracy of semantic parses, allowing for faster decoding . |
| Approach: | They propose an iterative method to extract code idioms from large source code corpora . they use most-frequent subtrees of their syntax trees to train semantic parsers to apply them . |
| Outcome: | The proposed method improves the state-of-the-art semantic parsers' accuracy and training time by more than 50%. |
Mapping Language to Code in Programmatic Context (D18-1)
Copied to clipboard
| Challenge: | Existing approaches for automatically mapping natural language to executable code have considered limited language or code environments. |
| Approach: | They propose a task of generating class member functions given English documentation and the programmatic context provided by the rest of the class. |
| Outcome: | The proposed model can generate member functions from documentation and the class environment. |