Papers by Christian Bartelt
A Mechanistic Analysis of a Transformer Trained on a Symbolic Multi-Step Reasoning Task (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies do not provide insights into the internal mechanisms driving the observed abilities. |
| Approach: | They propose to implement a depth-bounded recurrent mechanism that operates in parallel and stores intermediate results in selected token positions. |
| Outcome: | The proposed model implements a depth-bounded recurrent mechanism that operates in parallel and stores intermediate results in selected token positions. |
Large Language Models Share Representations of Latent Grammatical Concepts Across Typologically Diverse Languages (2025.naacl-long)
Copied to clipboard
| Challenge: | In the brains of human bilinguals, syntax processing may occur in similar regions for their first and second language, depending on factors like when the second language was learned and language proficiency. |
| Approach: | They propose to use sparse autoencoders to train Llama-3-8B and Aya-23-8B models to train multilingual models that share morphsyntactic representations of grammatical concepts. |
| Outcome: | The proposed model can predict plural verbs in different languages by activating the same plural feature. |