Papers by Chengyue Gong
LanguageFlow: Advancing Diffusion Language Generation with Probabilistic Flows (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent work has demonstrated success in controlling sentence attributes and structure based on diffusion language models. |
| Approach: | They propose a language-rectified flow method that reformulates standard probabilistic flow models to learn ordinary differential equations to transport between the source and target distributions. |
| Outcome: | The proposed method outperforms baselines on three fine-grained control tasks and multiple high-quality text editing tasks. |
Harmless Transfer Learning for Item Embeddings (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to learn item embeddings for categorical features are limited by the frequency of items in real-world. |
| Approach: | They propose a method that transfers knowledge from frequent items to rare items by introducing an auxiliary transfer loss. |
| Outcome: | The proposed framework significantly boosts the performance on a variety of NLP and recommendation system tasks. |
SAFER: A Structure-free Approach for Certified Robustness to Adversarial Word Substitutions (2020.acl-main)
Copied to clipboard
| Challenge: | State-of-the-art NLP models can be fooled by human-unaware transformations such as synonymous word substitution. |
| Approach: | They propose a method that constructs a stochastic ensemble by applying random word substitutions on the input sentences and leverages the statistical properties to provably certify the robustness. |
| Outcome: | The proposed method outperforms state-of-the-art methods on IMDB and Amazon text classification tasks with practically meaningful certified accuracy. |
Learning with Different Amounts of Annotation: From Zero to Many Labels (2021.emnlp-main)
Copied to clipboard
| Challenge: | a lack of annotator agreement can hinder training of NLP systems . we propose a learning algorithm that can learn from training examples with zero, one, or multiple labels. |
| Approach: | They propose an annotation distribution scheme that assigns multiple labels to training examples . they propose a learning algorithm that can learn from training examples with different amount of annotation . |
| Outcome: | The proposed method achieves consistent gains in two tasks, suggesting distributing labels unevenly among training examples can be beneficial for many NLP tasks. |
ALLSH: Active Learning Guided by Local Sensitivity and Hardness (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies show that labeling in crowdsourcing annotations is not an annotation artifact but rather a core linguistic phenomenon. |
| Approach: | They propose to retrieve unlabeled data with a local sensitivity and hardness-aware acquisition function. |
| Outcome: | The proposed method achieves consistent gains over the commonly used active learning strategies in various classification tasks. |
Passage-Mask: A Learnable Regularization Strategy for Retriever-Reader Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Xu et al., 2021) find that retrieval-reader models overfit top-rank passages and fail to reason over entire retrieval passages. |
| Approach: | They propose a passage mask mechanism which desensitizes the impact from top-rank retrieval passages and prevents the model from overfitting. |
| Outcome: | Experiments on open question answering, dialogue conversation, and fact verification show that the proposed model outperforms baselines. |
Knowing More About Questions Can Help: Improving Calibration in Question Answering (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing work on calibration focuses on model confidence, such as the max probability of the predicted class. |
| Approach: | They propose a calibration method which estimates whether model correctly predicts answer for each question. |
| Outcome: | The proposed calibration method achieves 5-10% gains on reading comprehension benchmarks. |