Papers by Yuqing Sun
End-to-end Adversarial Sample Generation for Data Augmentation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for generating adversarial samples have deceived many neural inference models, such as text classification and machine translation. |
| Approach: | They propose an adversarial sample generator that consists of a conditioned paraphrasing model and a condition generator and introduce a pretrained discriminator to help the adversarial sample generator adapt to the data characteristics. |
| Outcome: | The proposed approach improves the performance of the trained model on several tasks and is robust for various attacking techniques. |
Rectified Sparse Attention for Efficient Long-Sequence Generation (2026.findings-acl)
Copied to clipboard
Yutao Sun, Tianzhu Ye, Li Dong, Yuqing Xia, Jian Chen, Yizhao Gao, Shijie Cao, Jianyong Wang, Furu Wei
| Challenge: | Recent sparse decoding methods improve efficiency but suffer from KV cache misalignment, resulting in performance degradation. |
| Approach: | They propose a method that combines block-sparse attention with periodic dense rectification to bound error accumulation and preserve alignment with the pretraining distribution. |
| Outcome: | Experiments on math reasoning, language modeling, and retrieval tasks show that ReSA achieves near-lossless generation quality with significantly improved efficiency. |
Inducing Argument Facets for Faithful Opinion Summarization (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Faithful opinion summarization task involves generating a summary that covers the majority and minority opinions in documents. |
| Approach: | They propose a facets-guided opinion summarization method that induces facets and partitions documents into multiple facet-specific sets. |
| Outcome: | The proposed method outperforms state-of-the-art methods and multiple LLMs on two representative datasets and shows it can be used in specialty domains. |
Applying Contrastive Learning to Code Vulnerability Type Classification (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent approaches to classification of vulnerabilities ignore their relationships and treat each class in isolation, resulting in non-scalable code vector representations. |
| Approach: | They propose a hierarchical contrastive learning framework to bring vector representations of related CWEs closer together and use max-pooling to enable the model to handle longer vulnerability code inputs. |
| Outcome: | The proposed framework outperforms state-of-the-art methods by 2.97%-17.90% on accuracy and 0.98%-22.27% on weighted-F1 with even better performance on higher-quality datasets. |
CoLLiE: Collaborative Training of Large Language Models in an Efficient Way (2023.emnlp-demo)
Copied to clipboard
Kai Lv, Shuo Zhang, Tianle Gu, Shuhao Xing, Jiawei Hong, Keyu Chen, Xiaoran Liu, Yuqing Yang, Honglin Guo, Tengxiao Liu, Yu Sun, Qipeng Guo, Hang Yan, Xipeng Qiu
| Challenge: | Large language models (LLMs) are increasingly pivotal in a wide range of tasks . however, the resources required for training these models necessitate efficient solutions . |
| Approach: | They propose a library that facilitates collaborative training of large language models . they use 3D parallelism, parameter-efficient fine-tuning methods and optimizers . |
| Outcome: | The proposed library has proven superior training efficiency in comparison with prevalent solutions in pre-training and fine-tuning scenarios. |