Papers by Shimao Zhang
Getting More from Less: Large Language Models are Good Spontaneous Multilingual Learners (2024.emnlp-main)
Copied to clipboard
Shimao Zhang, Changjiang Gao, Wenhao Zhu, Jiajun Chen, Xin Huang, Xue Han, Junlan Feng, Chao Deng, Shujian Huang
| Challenge: | Large Language Models (LLMs) have shown impressive language capabilities, but most of them have very unbalanced performance across different languages. |
| Approach: | They propose to use question translation data to enhance LLMs' multilingual capabilities by using mechanistic interpretability methods. |
| Outcome: | The proposed method improves multilingual alignment even with unannotated answers in English and a wide range of languages even with instruction-tuned LLMs. |
MoRE: A Mixture of Low-Rank Experts for Adaptive Multi-Task Learning (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have revolutionized various domains, offering unprecedented performance across numerous tasks. |
| Approach: | They propose a new Mixture of Low-Rank Experts (MoRE) for multi-task PEFT to improve performance of LLMs with fewer parameters. |
| Outcome: | The proposed method improves performance over multiple tasks and no additional inference cost. |
Process-based Self-Rewarding Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to reward LLMs' outputs are not effective in mathematical reasoning scenarios and may lead to a decline in performance. |
| Approach: | They propose a process-based self-rewarding pipeline that integrates long-thought reasoning, step-wise LLM-as-a-Judge, and step- wise preference optimization within the existing paradigm. |
| Outcome: | The proposed model improves the performance of Large Language Models on multiple mathematical reasoning benchmarks and shows that it can surpass human capabilities. |