Papers with LAPE
Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Models (2024.acl-long)
Copied to clipboard
Tianyi Tang, Wenyang Luo, Haoyang Huang, Dongdong Zhang, Xiaolei Wang, Xin Zhao, Furu Wei, Ji-Rong Wen
| Challenge: | Despite the impressive multilingual capabilities demonstrated by LLMs, the understanding of how these abilities develop and function remains nascent. |
| Approach: | They propose a novel detection method to pinpoint language-specific neurons within LLMs by selectively activating or deactivating these neurons. |
| Outcome: | The proposed method can “steer” the output language of LLMs by selectively activating or deactivating language-specific neurons. |
Multilingual Language Models Encode Script Over Linguistic Structure (2026.acl-long)
Copied to clipboard
| Challenge: | a recent study suggests that multilingual language models organize representations around surface form, but the nature of this internal organization remains elusive. |
| Approach: | They analyze language-associated units across different model families and scales . romanization induces near-disjoint representations that align with neither native-script inputs nor English . |
| Outcome: | The results show that multilingual language models organize representations around surface form . romanization induces near-disjoint representations that align with neither native-script inputs nor English . |