Papers by Marcell Fekete
Linguistically Grounded Analysis of Language Models using Shapley Head Values (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for probing language models for morphosyntactic constructions are not well understood . language models gain knowledge of grammatical phenomena during pretraining, but exactly how this knowledge is encoded is not well established. |
| Approach: | They propose a method for probing language models via Shapley Head Values . they use a BLiMP dataset to test their method on linguistic constructions based on a Shaply Head Value method . |
| Outcome: | The proposed method can be used to investigate linguistic knowledge in language models . it shows that attention heads responsible for processing related linguistic phenomena cluster together . |
Limited-Resource Adapters Are Regularizers, Not Linguists (2025.acl-short)
Copied to clipboard
Marcell Fekete, Nathaniel Romney Robinson, Ernests Lavrinovics, Djeride Jean-Baptiste, Raj Dabre, Johannes Bjerva, Heather Lent
| Challenge: | Existing studies show that cross-lingual transfer from high-resource languages is promising for low-resourced machine translation. |
| Approach: | They propose to use adapter souping and cross-attention fine-tuning to leverage language transfer for Creoles, an under-served group of low-resource languages. |
| Outcome: | The proposed method improves performance over baselines but not meaningfully with adapters. |