Papers by Changhao Guan
Multi-Stage LLM Fine-Tuning with a Continual Learning Setting (2025.findings-naacl)
Copied to clipboard
Changhao Guan, Chao Huang, Hongliang Li, You Li, Ning Cheng, Zihe Liu, Yufeng Chen, Jinan Xu, Jian Liu
| Challenge: | Large language models (LLMs) have made significant progress in knowledge-intensive applications, but they may face a multi-stage continuous learning scenario. |
| Approach: | They propose a multi-stage continuous learning paradigm that includes a preference-based learning bias to identify potential knowledge conflicts and a self-distillation-based data augmentation strategy to expand and enrich the training corpus. |
| Outcome: | The proposed learning paradigm achieves a significant improvement in accuracy after 7 stages of fine-tuning compared to previous methods while preserving general knowledge. |
Novel Slot Detection With an Incremental Setting (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Current dialogue systems face diverse user requests and rapid change domains, making quickly adapt to scenarios with previous unseen slot types becomes a major challenge. |
| Approach: | They propose an incremental novel slot detection task which separates the dialogue system to deal with novel types as two major phrases: 1) model discovers unknown slots; 2) training model to possess the capability to handle new classes. |
| Outcome: | The proposed approach overcomes catastrophic forgetting during the process of INSD and is highly effective. |
Boosting Data Utilization for Multilingual Dense Retrieval (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies focus on fine-tuning multilingual dense retrieval models, but data scarcity for low-resource languages makes it difficult to align representations in a shared vector space. |
| Approach: | They propose to obtain high-quality hard negative samples and effective mini-batch data to boost data utilization for multilingual dense retrieval by obtaining high-quality negative samples. |
| Outcome: | The proposed method outperforms existing baselines on a multilingual retrieval benchmark, MIRACL, with 16 languages. |
Multilingual Collaborative Defense for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing safeguards for Large Language Models are vulnerable to "jailbreaking" harmful queries. |
| Approach: | They propose a learning method that optimizes a continuous soft safety prompt automatically to facilitate multilingual safeguarding of LLMs. |
| Outcome: | The proposed method outperforms previous approaches in multilingual jailbreak defense while exhibiting strong cross-lingual generalization. |
DoRA: Enhancing Parameter-Efficient Fine-Tuning with Dynamic Rank Distribution (2024.acl-long)
Copied to clipboard
| Challenge: | Existing parameter-efficient fine-tuning methods such as Low-Rank Adaptation ignore the differential parameter budget requirements across weight matrices, which may lead to suboptimal fine-uning outcomes. |
| Approach: | They propose a parameter-efficient low-rank Adaptation method that decomposes high-rank LoRA layers into structured single-rank components and allows dynamic pruning of parameter budget . |
| Outcome: | The proposed method outperforms LoRA and LoRA with the same parameter budget and performance. |