Papers by Shengjie Qiu
Incremental Sequence Labeling: A Tale of Two Shifts (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to incremental sequence labeling have focused on the E2O and O2E issues, but neglect the O2e issue. |
| Approach: | They propose a framework for incremental sequence labeling without semantic shifts that mitigate catastrophic forgetting in models by using knowledge distillation to maintain the model’s discriminative ability for old entities. |
| Outcome: | The proposed framework mitigates catastrophic forgetting in models while maintaining discriminative ability for old entities while minimizing the model’s bias towards new entities. |
Preserving Commonsense Knowledge from Pre-trained Language Models via Causal Inference (2023.acl-long)
Copied to clipboard
Junhao Zheng, Qianli Ma, Shengjie Qiu, Yue Wu, Peitian Ma, Junlong Liu, Huawen Feng, Xichen Shang, Haibin Chen
| Challenge: | Existing studies attribute catastrophic forgetting to fine-tuning, and they retain pre-trained knowledge indiscriminately without identifying what knowledge is transferable. |
| Approach: | They propose a unified objective for fine-tuning to retrieve the causality back from pre-trained data and use it to mitigate negative transfer while preserving knowledge. |
| Outcome: | The proposed method outperforms state-of-the-art fine-tuning methods on commonsense QA datasets and can be implemented as a plug-in module to inflate the performance of existing QA models. |