Papers by Luyang Liu
Heterogeneous LoRA for Federated Fine-tuning of On-Device Foundation Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | federated fine-tuning of ODFMs is limited due to their limited size and system heterogeneity . emerging foundation models (FMs) have remarkable zero/few shot learning capabilities . |
| Approach: | They propose a federated fine-tuning method that leverages system and data heterogeneity at the edge. |
| Outcome: | a proposed method for federated fine-tuning improves performance on ODFMs . it allows heterogeneous LoRA ranks across clients for their individual system resources . |
Task-oriented Word Embedding for Text Classification (C18-1)
Copied to clipboard
| Challenge: | Existing word embeddings only consider contextual information, which is suboptimal when used in various tasks due to a lack of task-specific features. |
| Approach: | They propose a task-oriented word embedding method that regularizes the distribution of words to enable a clear classification boundary. |
| Outcome: | The proposed method outperforms the state-of-the-art methods on a text classification task. |
Learning LLM Preference over Intra-Dialogue Pairs: A Framework for Utterance-level Understandings (2025.naacl-industry)
Copied to clipboard
Xuanqing Liu, Luyang Kong, Wei Niu, Afshin Khashei, Belinda Zeng, Steve Johnson, Jon Jay, Davor Golac, Matt Pope
| Challenge: | Large language models (LLMs) have demonstrated remarkable capabilities in handling complex dialogue tasks without requiring use case-specific fine-tuning. |
| Approach: | They propose a framework that combines the scalability of LLM-generated labels with the precision of human annotations to achieve higher speed and accuracy comparable to larger models. |
| Outcome: | The proposed framework significantly improves accuracy across utterance-level dialogue tasks, including sentiment detection (over 2%), dialogue act classification (over 1.5%), etc. |
MADAWSD: Multi-Agent Debate Framework for Adversarial Word Sense Disambiguation (2025.emnlp-main)
Copied to clipboard
Kaiyuan Zhang, Qian Liu, Luyang Zhang, Chaoqun Zheng, Shuaimin Li, Bing Xu, Muyun Yang, Xinxiao Qiao, Wenpeng Lu
| Challenge: | Word sense disambiguation (WSD) is a fundamental yet challenging task in natural language processing. |
| Approach: | a novel multi-agent Debate framework for adversarial word Sense disambiguation is proposed . the framework simulates a real-world debate environment where multiple agents engage in discussions about ambiguous words in the context of adversarials. |
| Outcome: | The proposed framework integrates with existing LLMs and improves models in Chinese language . it shows that it can be used to improve models in the Chinese language and improve performance . |
DU-VLG: Unifying Vision-and-Language Generation via Dual Sequence-to-Sequence Pre-training (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing vision-and-language generation models cannot utilize pair-wise images and text through bi-directional generation due to the limitations of the model structure and pre-training objectives. |
| Approach: | They propose a framework which unifies vision-and-language generation as sequence generation problems. |
| Outcome: | The proposed framework achieves better performance than variants trained with uni-directional generation objectives or the variant without the commitment loss on image captioning and text-to-image generation datasets. |
BPID: A Benchmark for Personal Identity Deduplication (2024.emnlp-industry)
Copied to clipboard
Runhui Wang, Yefan Tao, Adit Krishnan, Luyang Kong, Xuanqing Liu, Yuqian Deng, Yunzhao Yang, Henrik Johnson, Andrew Borthwick, Shobhit Gupta, Aditi Gundlapalli, Davor Golac
| Challenge: | Data deduplication is a critical task in data management and mining, focused on consolidating duplicate records that refer to the same entity. |
| Approach: | They propose to use a dataset with 1,000,000 unlabeled synthetic PII profiles and a subset of 10,000 pairs curated and labeled as matches or non-matches. |
| Outcome: | The proposed datasets contain synthetic profiles built from publicly available sources that do not represent real individuals. |