Papers by Hantian Ding
A Static Evaluation of Code Completion by Large Language Models (2023.acl-industry)
Copied to clipboard
Hantian Ding, Varun Kumar, Yuchen Tian, Zijian Wang, Rob Kwiatkowski, Xiaopeng Li, Murali Krishna Ramanathan, Baishakhi Ray, Parminder Bhatia, Sudipta Sengupta
| Challenge: | Large language models trained on code have shown great potential to increase productivity of software developers. |
| Approach: | They propose a static evaluation framework to quantify static errors in Python code completions by leveraging Abstract Syntax Trees. |
| Outcome: | The proposed framework is more efficient and applicable to code in the wild. |
Towards Open-Domain Topic Classification (2022.naacl-demo)
Copied to clipboard
| Challenge: | Existing supervised classification models are insensitive to class names, but are no longer effective in open-domain tasks where the taxonomy is unbounded. |
| Approach: | They propose a topic classification system that accepts user-defined taxonomy in real time . they train a pretrained language model on a new Wikipedia dataset and train it on Wikipedia . |
| Outcome: | The proposed system improves over existing zero-shot models and performs competitively with weakly-supervised models trained on in-domain data. |
SP-10K: A Large-scale Evaluation Set for Selectional Preference Acquisition (P19-1)
Copied to clipboard
| Challenge: | Selectional Preference (SP) is a common phenomenon in human language and has been shown to be useful in many natural language processing tasks. |
| Approach: | They propose a large-scale evaluation set that provides human ratings for the plausibility of 10,000 SP pairs over five SP relations, covering 2,500 most frequent verbs, nouns, and adjectives in American English. |
| Outcome: | The proposed evaluation sets provide human ratings for plausibility of 10,000 SP pairs over five SP relations covering 2,500 most frequent verbs, nouns, and adjectives in American English. |
Exploring Continual Learning for Code Generation Models (2023.acl-short)
Copied to clipboard
Prateek Yadav, Qing Sun, Hantian Ding, Xiaopeng Li, Dejiao Zhang, Ming Tan, Parminder Bhatia, Xiaofei Ma, Ramesh Nallapati, Murali Krishna Ramanathan, Mohit Bansal, Bing Xiang
| Challenge: | Large-scale code generation models such as Copilot and CodeT5 are expensive to train and re-train. |
| Approach: | They propose a benchmark for Continual Learning (CL) that covers a wide range of tasks with different input and output programming languages. |
| Outcome: | The proposed method improves on Prompt Pooling with Teacher Forcing, which suffers catastrophic forgetting due to stark distribution shifts in coding tasks. |
Planning-Aware Code Infilling via Horizon-Length Prediction (2025.emnlp-main)
Copied to clipboard
| Challenge: | Current approaches to fill-in-the-middle (FIM) often fail to generate content that aligns well with the surrounding context. |
| Approach: | They propose a training objective that teaches models to predict the number of remaining middle tokens at each step. |
| Outcome: | The proposed training objective improves FIM performance by up to 24% on diverse benchmarks across file-level and repository-level. |