Papers by Lei Sang
TalkLoRA: Communication-Aware Mixture of Low-Rank Adaptation for Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing LoRA methods assume that experts operate independently, leading to unstable routing, expert dominance. |
| Approach: | They propose a communication-aware MoELoRA framework that relaxes this assumption by introducing expert-level communication prior to routing. |
| Outcome: | The proposed framework outperforms vanilla LoRA and MoELoRA on diverse language understanding tasks while maintaining expert dominance. |
DICP: Deep In-Context Prompt for Event Causality Identification (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing prompt-learning-based methods concatenate in-context examples only at the input layer, limiting the model’s ability to capture abstract semantic cues necessary for identifying complex causal relationships. |
| Approach: | They propose a model that injects in-context examples into the deeper layer of a pre-trained language model (PLM) this model leverages hierarchical semantic representations formed in deeper layers, thereby enhancing its capacity to learn high-level causal abstractions. |
| Outcome: | The proposed model improves on two widely used datasets and shows that it can learn high-level causal abstractions. |
GraphLoRA: Structure-Aware Low-Rank Adaptation for Large Language Model Recommendation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for translating collaborative information into textual prompts or injecting pre-trained embeddings into the LLM treat structural information as static input and fail to capture high-order relational dependencies. |
| Approach: | They propose a framework that generalizes low-rank adaptation from independent to structure-aware propagation by embedding a trainable graph message-passing network within the low-ranked adaptation pathway. |
| Outcome: | Experiments on multiple benchmarks show that GraphLoRA outperforms state-of-the-art recommendation methods and achieves superior generalization. |