Papers by Clément Dumas
Separating Tongue from Thought: Activation Patching Reveals Language-Agnostic Concept Representations in Transformers (2025.acl-long)
Copied to clipboard
| Challenge: | A central question in multilingual language modeling is whether large language models develop a universal concept representation, disentangled from specific languages. |
| Approach: | They analyze latent representations during a word-translation task in transformer-based LLMs and extract the residual stream of the last token of the word to be translated and insert the mean at the corresponding positions in the forward pass. |
| Outcome: | The proposed model can translate a word in multiple languages without changing the language and vice versa. |