Papers by Jinjie Yuan
LoNAS: Elastic Low-Rank Adapters for Efficient Large Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) reach hundreds of billions of parameters and require resources for training and inference stages. |
| Approach: | They propose a low-rank adapter to reduce the number of trainable parameters in a model and reduce memory requirements. |
| Outcome: | The proposed approach reduces memory and compute requirements while preserving performance. |
Mamba-Shedder: Post-Transformer Compression for Efficient Selective Structured State Space Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Large pre-trained models have achieved outstanding results in sequence modeling . alternative architectures, such as Selective Structured State Space Models (SSMs), have been proposed to address these inefficiencies. |
| Approach: | They propose to reduce the size and computational overhead of large pre-trained models by removing selected components at different granularities. |
| Outcome: | The proposed models achieve a speedup of up to 1.4x during inference while maintaining accuracy. |
RTTC: Reward-Guided Collaborative Test-Time Compute (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Reward-Guided Test-Time Compute (RTTC) is a powerful paradigm for large language models . indiscriminate application of TTC strategy incurs substantial computational overhead . |
| Approach: | They propose a framework that adaptively selects the most effective TTC strategy for each query via a pretrained reward model. |
| Outcome: | The proposed framework maximizes accuracy across diverse domains and tasks. |
SQFT: Low-cost Model Adaptation in Low-precision Sparse Foundation Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large pre-trained models are often adapted to a desired domain or task through a fine-tuning stage. |
| Approach: | They propose an end-to-end solution for sparse parameter-efficient fine-tuning of large pre-trained models. |
| Outcome: | The proposed approach can be used to combine sparse weights with low-rank adapters without losing sparsity and accuracy. |
CPRM: A LLM-based Continual Pre-training Framework for Relevance Modeling in Commercial Search (2025.naacl-industry)
Copied to clipboard
Kaixin Wu, Yixin Ji, Zeyuan Chen, Qiang Wang, Cunxiang Wang, Hong Liu, Baijun Ji, Xu Jia, Zhongyi Liu, Jinjie Gu, Yuan Zhou, Linjian Mo
| Challenge: | Relevance modeling between queries and items is a key component of commercial search engines. |
| Approach: | They propose a framework for continual pre-training of LLMs to enhance domain knowledge . they employ queries and multi-field item to jointly pre-train for enhancing domain knowledge. |
| Outcome: | The proposed model achieves convincing performance compared to strong baselines. |