Papers by Cenyuan Zhang
Aligning Large Language Models with Human Preferences through Representation Engineering (2024.acl-long)
Copied to clipboard
Wenhao Liu, Xiaohua Wang, Muling Wu, Tianlong Li, Changze Lv, Zixuan Ling, Zhu JianHao, Cenyuan Zhang, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Existing methods for achieving this alignment involve employing reinforcement learning from human feedback (RLHF) Existing approaches involve using RLHF to fine-tune LLMs based on human labels . however, RLRF is susceptible to instability during fine- tuning and presents challenges in implementation. |
| Approach: | They propose to use reinforcement learning from human feedback to fine-tune large language models with human preferences to achieve precise control of model behavior. |
| Outcome: | Experiments show that RAHF can be used to capture and manipulate representations to align with a broad spectrum of human preferences or values rather than being confined to a single concept or function. |
Exploration and Exploitation: Two Ways to Improve Chinese Spelling Correction Models (2021.acl-short)
Copied to clipboard
| Challenge: | Experimental results show that a sequence-to-sequence learning framework with neural networks can be effective for Chinese Spelling Correction (CSC) |
| Approach: | They propose a sequence-to-sequence learning framework with neural networks that generates more valuable training instances and adds task-specific examples to enhance the model. |
| Outcome: | The proposed method improves generalization and robustness of multiple CSC models across three datasets. |
Improving the Adversarial Robustness of NLP Models by Information Bottleneck (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have shown that adversarial examples can be directly attributed to the presence of non-robust features. |
| Approach: | They propose to capture task-specific robust features while eliminating non-robust ones . they show that models can achieve significant improvement in robust accuracy . |
| Outcome: | The proposed method outperforms all defense methods on SST-2, AGNEWS and IMDB datasets while achieving no performance drop. |
Advancing Parameter Efficiency in Fine-tuning via Representation Editing (2024.acl-long)
Copied to clipboard
Muling Wu, Wenhao Liu, Xiaohua Wang, Tianlong Li, Changze Lv, Zixuan Ling, Zhu JianHao, Cenyuan Zhang, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Parameter Efficient Fine-Tuning (PEFT) has gained significant attention for its ability to achieve competitive results while updating only a small subset of trainable parameters. |
| Approach: | They propose a new approach to fine-tuning neural models that scales and biases the representation produced at each layer. |
| Outcome: | The proposed approach reduces the number of trainable parameters by a factor of 25,700 compared to full parameter fine-tuning and by . 32 compared with LoRA. |
Watermarking PLMs on Classification Tasks by Combining Contrastive Learning with Weight Perturbation (2023.findings-emnlp)
Copied to clipboard
Chenxi Gu, Xiaoqing Zheng, Jianhan Xu, Muling Wu, Cenyuan Zhang, Chengsong Huang, Hua Cai, Xuanjing Huang
| Challenge: | Large pre-trained language models (PLMs) are highly valuable intellectual property due to their expensive training costs. |
| Approach: | They propose to embed backdoors that can be triggered by specific inputs into models by model watermarking. |
| Outcome: | The proposed method can be used to protect the intellectual property of large pre-trained language models without knowledge about downstream tasks. |
Towards Adversarially Robust Text Classifiers by Learning to Reweight Clean Examples (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing defense methods improve the adversarial robustness by making models adapt to training set augmented with some adversarials. |
| Approach: | They propose to introduce a reweighting mechanism to calibrate the training distribution to obtain robust models. |
| Outcome: | The proposed method minimizes the loss of validation set mixed with clean examples and adversarial ones in an online learning manner. |
Promoting Data and Model Privacy in Federated Learning through Quantized LoRA (2024.findings-emnlp)
Copied to clipboard
Zhu JianHao, Changze Lv, Xiaohua Wang, Muling Wu, Wenhao Liu, Tianlong Li, Zixuan Ling, Cenyuan Zhang, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Existing federated learning frameworks require substantial data and computational resources to develop large language models. |
| Approach: | They propose a method that distributes a quantized version of the model’s parameters during training and combine it with a popular fine-tuning method to significantly reduce communication costs. |
| Outcome: | The proposed method enables accurate estimations for parameter updates while preventing clients from accessing a model whose performance is comparable to the centrally hosted one. |