Papers by Bastian Bunzeck
Subword models struggle with word learning, but surprisal hides it (2025.acl-short)
Copied to clipboard
| Challenge: | Subword LMs struggle to discern words and non-words with high accuracy, character LM models do this easily and consistently. |
| Approach: | They propose to model word learning in subword and character language models with the psycholinguistic lexical decision task. |
| Outcome: | The results suggest that word learning and syntactic learning are separable in character LMs. |
Small Language Models Also Work With Small Vocabularies: Probing the Linguistic Abilities of Grapheme- and Phoneme-Based Baby Llamas (2025.coling-main)
Copied to clipboard
| Challenge: | Existing studies on LMs have focused on linguistic generalizations and representations from developmentally plausible data. |
| Approach: | They propose to use phoneme- and grapheme-based language models to learn linguistic units at and below the word level. |
| Outcome: | The proposed models can achieve strong performance on syntactic and novel benchmarks and match grapheme-based models in standard tasks and novel evaluations. |
BabyBabelLM: A Multilingual Benchmark of Developmentally Plausible Training Data (2026.eacl-long)
Copied to clipboard
Jaap Jumelet, Abdellah Fourtassi, Akari Haga, Bastian Bunzeck, Bhargav Shandilya, Diana Galvan-Sosa, Faiz Ghifari Haznitrama, Francesca Padovani, Francois Meyer, Hai Hu, Julen Etxaniz, Laurent Prevot, Linyang He, María Grandury, Mila Marcheva, Negar Foroutan, Nikitas Theodoropoulos, Pouya Sadeghi, Siyuan Song, Suchir Salhan, Susana Zhou, Yurii Paniv, Ziyin Zhang, Arianna Bisazza, Alex Warstadt, Leshem Choshen
| Challenge: | prevailing trend in language modeling research is to prioritize scaling, authors say . from infancy to maturity, English learners acquire language through exposure to less than 100M words . |
| Approach: | They propose a multilingual collection of datasets modeling the language a person observes from birth until they acquire a native language. |
| Outcome: | The proposed models outperform models trained on a fixed, developmentally plausible English corpus on various benchmarks. |