Papers by Tongyao Zhu
Beyond Memorization: The Challenge of Random Memory Access in Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Recent advances in Language Models (LMs) have shown their effectiveness in knowledge-intensive tasks. |
| Approach: | They investigate whether a generative language model is able to access its memory sequentially or randomly. |
| Outcome: | The proposed LMs are able to access memory sequentially or randomly. |
When Does Mixing Help? Analyzing Query Embedding Interpolation in Multilingual Dense Retrieval (2026.acl-long)
Copied to clipboard
| Challenge: | Existing multilingual retrievers are largely untested for mixed language queries . sensitivity of dense retrievers to mixed language querying remains poorly understood . |
| Approach: | They propose to use mixed queries as an interpolation of monolingual embeddings to evaluate retrieval performance. |
| Outcome: | The proposed model outperforms the best monolingual endpoint in 88/105 cases. |
How Can Synthetic Data Improve Multilingual Language Model Pretraining? A Data Quality Perspective (2026.acl-long)
Copied to clipboard
| Challenge: | Low-resource languages are a long-tail problem for multilingual LLMs due to limited high-quality training data. |
| Approach: | They propose a method that translates high-quality, knowledge-rich English data into low-resource languages . they propose SynRank, which leverages synthetic data as positive samples to train a classifier . |
| Outcome: | The proposed method matches handcrafted rule-based filtering by human experts and significantly improves knowledge-intensive tasks with less data. |