Papers by Bao Rong
Orthogonal Subspace Learning for Language Model Continual Learning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for continual learning in language models suffer catastrophic forgetting when learning sequential tasks. |
| Approach: | They propose an orthogonal low-rank adaptation approach for continual learning in language models that uses orthogons to learn sequentially. |
| Outcome: | The proposed approach outperforms state-of-the-art methods on continual learning benchmarks and preserves generalization ability of LLMs on unseen tasks. |
Flooding-X: Improving BERT’s Resistance to Adversarial Attacks via Loss-Restricted Fine-Tuning (2022.acl-long)
Copied to clipboard
Qin Liu, Rui Zheng, Bao Rong, Jingyi Liu, ZhiHua Liu, Zhanzhan Cheng, Liang Qiao, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Existing approaches to generating adversarial perturbations scale up the cost of training computational complexity by the number of gradient steps it takes to obtain the adversarials. |
| Approach: | They propose a flood method which aims at better generalization and a criterion to bring hyper-parameter-dependent flooding into effect with a narrowed-down search space by measuring how the gradient steps taken within one epoch affect the loss of each batch. |
| Outcome: | The proposed method improves BERT’s resistance to textual adversarial attacks by a large margin and achieves state-of-the-art robust accuracy on various text classification and GLUE tasks. |
Fixing Distribution Shifts of LLM Self-Critique via On-Policy Self-Play Training (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models show impressive performance in a wide range of linguistic tasks, but their performance on complex reasoning tasks is still signif-icantly lower than the human level. |
| Approach: | They propose a reinforcement learning framework to synchronize the reasoning and critique capabilities of language models by using Monte Carlo sampling to give appropriate rewards to the model's critique content. |
| Outcome: | The proposed framework improves the model's reasoning and critique capabilities by 5.40 and 3.66 points, respectively, compared to the best baseline approach. |
PlugAT: A Plug and Play Module to Defend against Textual Adversarial Attack (2022.coling-1)
Copied to clipboard
| Challenge: | Recent studies have shown that adversarial examples can be easily fooled by adversarially perturbed examples. |
| Approach: | They propose a pluggable defense module PlugAT to provide robust predictions by adding a few trainable parameters to the model inputs while keeping the original model frozen. |
| Outcome: | The proposed model improves robustness over several strong baselines whilst training only 9.1% parameters. |
Robust Lottery Tickets for Pre-trained Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that pre-trained language models contain smaller matching subnetworks that are not robust to adversarial examples. |
| Approach: | They propose a method to find robust tickets hidden in pre-trained language models by learning binary weight masks and an adversarial loss objective to guide the search. |
| Outcome: | The proposed method improves on previous work on adversarial robustness evaluation. |
CASN:Class-Aware Score Network for Textual Adversarial Detection (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to counteract adversarial attacks can be divided into two directions, adversarials defense and adversarially detection. |
| Approach: | They propose a score-based generative method to implicitly model the data distribution using a log-density distribution and supervised contrastive learning to guide the estimation using label information. |
| Outcome: | The proposed method improves on three text classification tasks on four advanced attack algorithms. |
CEDAR: A Chinese Evaluation Dataset for Computational Argumentation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing debate datasets neglect important labels for argument mining, generation, and evaluation. |
| Approach: | They propose a Chinese Evaluation Dataset for Computational Argumentation that includes key arguments and key rhetorical figures, debater roles, modal words, debate results and transcripts. |
| Outcome: | The proposed dataset covers 600 debates about 318 topics from Chinese debate competitions. |