Papers by Yu-Kun Li
Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models (2026.acl-long)
Copied to clipboard
Xin Cheng, Wangding Zeng, Damai Dai, Qinyu Chen, Bingxuan Wang, Zhenda Xie, Kezhao Huang, Xingkai Yu, Zhewen Hao, Han Zhang, Yu-Kun Li, Huishuai Zhang, Dongyan Zhao, Wenfeng Liang
| Challenge: | Mixture-of-Experts (MoE) scales capacity via conditional computation, but lacks knowledge lookup primitive. |
| Approach: | They propose a conditional memory instantiated via Deep Sparse Embedding (DSE) they propose 'u-shaped scaling law' that identifies optimal balance between MoE experts and DSE memory . |
| Outcome: | The proposed model outperforms an iso-parameter and isoFLOPs MoE baseline across knowledge and reasoning benchmarks and is infrastructure-efficient. |
ERNIE-Gram: Pre-Training with Explicitly N-Gram Masked Language Modeling for Natural Language Understanding (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to model coarse-grained linguistic information do not integrate coarse-gram information into pre-training. |
| Approach: | They propose an explicitly n-gram masking method to enhance integration of coarse-grained linguistic information into pre-training. |
| Outcome: | The proposed method outperforms existing models on English and Chinese text corpora and fine-tunes on 19 downstream tasks. |