Papers by Yinggui Wang
Mitigating Catastrophic Forgetting in Large Language Models with Forgetting-aware Pruning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have shown impressive capabilities in various downstream tasks but typically face Catastrophic Forgetting (CF) during fine-tuning. |
| Approach: | They propose a pruning-based approach to balance CF and downstream task performance by integrating the ratio of the task vector to pre-trained model parameters into the pruning criteria. |
| Outcome: | The proposed pruning-based approach limits CF to just 0.25% while maintaining 99.67% accuracy on downstream tasks. |
GradOT: Training-free Gradient-preserving Offsite-tuning for Large Language Models (2025.acl-long)
Copied to clipboard
Kai Yao, Zhaorui Tan, Penglei Gao, Lichun Li, Kaixin Wu, Yinggui Wang, Yuan Zhao, Yixin Ji, Jianke Zhu, Wei Wang
| Challenge: | Existing methods for offsite-tuning of large language models require high computational costs and lack theoretical analysis. |
| Approach: | They propose an offsite-tuning approach that selectively applies compression techniques such as rank compression and channel pruning to preserve the gradients of fine-tuned adapters while ensuring privacy. |
| Outcome: | The proposed method surpasses existing OT methods in privacy protection and model performance. |
TaiChi: Improving the Robustness of NLP Models by Seeking Common Ground While Reserving Differences (2024.lrec-main)
Copied to clipboard
| Challenge: | Pre-trained Language Models are vulnerable to adversarial examples that introduce human-imperceptible perturbations to clean examples to deceive the models. |
| Approach: | They propose a Siamese network-based approach to teach adversarial models to focus on similarities . they propose combining two sub-networks sharing the same structure but trained on clean and adversarials . |
| Outcome: | The proposed approach reduces the differences between clean and adversarial samples and focuses more on similarities. |
DaMoC: Efficiently Selecting the Optimal Large Language Model for Fine-tuning Domain Tasks Based on Data and Model Compression (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models excel in general tasks but struggle with domain-specific ones, requiring fine-tuning with specific data. |
| Approach: | They propose a Data and Model Compression Framework that categorizes data filtering methodologies into three distinct paradigms: (1) distribution-aware methods, (2) quality-a aware methods, and (3) hybrid approaches considering both dimensions. |
| Outcome: | The proposed framework can select the optimal LLM while saving approximately 20-fold in training time. |
CoTrust: Privacy-Preserving Collaboration Between Large and Small Language Models in Trusted Execution Environments (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) provide powerful text generation capabilities, but accessing sensitive user inputs raises privacy concerns. |
| Approach: | They propose a privacy-preserving collaborative inference framework that combines large language models with small language models inside TEE to preserve privacy. |
| Outcome: | Experiments show that CoTrust outperforms unconstrained LLMs on multiple question answering and summarization benchmarks while maintaining strong privacy protection. |
A Fully Probabilistic Perspective on Large Language Model Unlearning: Evaluation and Optimization (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Model Unlearning (LLMU) is a promising way to remove private or sensitive information from large language models. |
| Approach: | They propose a Fully Probabilistic Evaluation framework that incorporates input and output distributions in LLMU evaluation. |
| Outcome: | The proposed framework improves unlearning effectiveness by 50.1% and robustness by 37.2% on Llama-2-7B. |
Privacy Evaluation Benchmarks for NLP Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Several kinds of privacy attacks are studied in depth, but they are non-systematic and lack a comprehensive understanding of the impact caused by the attacks. |
| Approach: | They propose a privacy attack and defense evaluation benchmark in the field of NLP . they propose an improved attack method and a chained framework for privacy attacks . |
| Outcome: | The proposed framework can be chained to achieve a higher-level attack objective. |
DPF-CM: A Data Processing Framework with Privacy-Preserving Vector Databases for Chinese Medical LLMs Training and Deployment (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new framework for training data processing for Chinese medical language models is proposed . experimental results show that the framework significantly improves model accuracy . |
| Approach: | They propose a data processing framework for Chinese medical language models training and deployment . the framework is based on a question-oriented model training strategy and privacy preservation . |
| Outcome: | The proposed framework significantly improves model accuracy and reduces privacy leakage by 27%. |