Papers by Catherine Arnett
CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data (2026.acl-long)
Copied to clipboard
Pedro Ortiz Suarez, Laurie Burchell, Catherine Arnett, Rafael Mosquera, Sara Hincapié Monsalve, Thom Vaughan, Damian Stewart, Malte Ostendorff, Idris Abdulmumin, Vukosi Marivate, Shamsuddeen Hassan Muhammad, Atnafu Lambebo Tonja, Hend Al-Khalifa, Nadia Ghezaiel Hammouda, Verrah Akinyi Otiende, Tack Hwa Wong, Jakhongir Saydaliev, Melika Nobakhtian, Muhammad Ravi Shulthan Habibi, Chalamalasetti Kranti, Carol Muchemi, Khang Nguyen, Faisal Muhammad Adam, Luis Frentzen Salim, Reem Alqifari, Cynthia Jayne Amol, Joseph Marvin Imperial, Ilker Kesen, Ahmad Mustafid, Pavel Stepachev, Leshem Choshen, David Anugraha, Hamada Nayel, Seid Muhie Yimam, Vallerie Alexandra Putra, My Chiffon Nguyen, Azmine Toushik Wasi, Gouthami Vadithya, Rob Van Der Goot, Lanwenn ar C’horr, Karan Dua, Andrew Yates, Mithil Bangera, Yeshil Bangera, Hitesh Laxmichand Patel, Shu Okabe, Fenal Ashokbhai Ilasariya, Dmitry Gaynullin, Genta Indra Winata, Yiyuan Li, Juan Pablo Martínez, Amit Agarwal, Ikhlasul Akmal Hanif, Raia Abu Ahmad, Esther Adenuga, Filbert Aurelian Tjiaranata, Weerayut Buaphet, Michael Anugraha, Sowmya Vajjala, Benjamin L Rice, Azril Hafizi Amirudin, Jesujoba Oluwadara Alabi, Srikant Panda, Yassine Toughrai, Bruhan Kyomuhendo, Daniel Ruffinelli, null Akshata, Manuel Goulão, Ej Zhou, Ingrid Gabriela Franco Ramirez, Cristina Aggazzotti, Konstantin Dobler, Jun Kevin, Quentin Pagès, Nicholas Andrews, Nuhu Ibrahim, Mattes Ruckdeschel, Amr Keleg, Mike Zhang, Casper Rufaro Muziri, Saron Samuel, Sotaro Takeshita, Kun Kerdthaisong, Luca Foppiano, Rasul Dent, Tommaso Green, Ahmad Mustapha Wali, Kamohelo Makaaka, Vicky Feliren, Inshirah Idris, Hande Celikkanat, Abdulhamid Abubakar, Jean Maillard, Benoît Sagot, Thibault Clérice, Kenton Murray, Sarah K. K. Luger
| Challenge: | Language identification (LID) is a fundamental step in curating multilingual corpora. |
| Approach: | They introduce CommonLID, a community-driven, human-annotated LID benchmark for the web domain, covering 109 languages. |
| Outcome: | The proposed benchmark covers 109 languages and shows that existing evaluations overestimate accuracy for many languages in the web domain. |
Structural Priming Demonstrates Abstract Grammatical Representations in Multilingual Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Abstract grammatical knowledge is key to linguistic generalization in humans . strong evidence for grammatikal abstraction in humans comes from structural priming . |
| Approach: | They compare human models of crosslingual structural priming to human models . they find evidence for abstract monolingual and crosslingual grammatical representations . |
| Outcome: | The results show that grammatical representations in multilingual models are similar to humans . the strongest evidence for grammatikal abstraction in humans comes from structural priming . |
On the Acquisition of Shared Grammatical Representations in Bilingual Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Crosslingual transfer is crucial to contemporary language models’ multilingual capabilities, but how it occurs is not well understood. |
| Approach: | They use structural priming to study grammatical representations in humans by controlling for training data quantity and language exposure. |
| Outcome: | The proposed model is able to learn a language in two languages and has a higher likelihood of learning a prepositional object (PO) dative sentence than a double object (DO) . |
When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource Languages (2024.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual language models are widely used to extend NLP systems to low-resource languages. |
| Approach: | They pre-train over 10,000 monolingual and multilingual language models for over 250 languages including multiple language families that are under-studied in NLP. |
| Outcome: | The results show that adding multilingual data improves low-resource language modeling performance, similar to increasing low-source dataset sizes by up to 33%. |
Why do language models perform worse for morphologically complex languages? (2025.coling-main)
Copied to clipboard
| Challenge: | Language models perform differently across languages, a new study suggests . morphological typology may explain some of the performance differences, authors say . |
| Approach: | They propose to test morphological alignment of tokenizers, tokenization quality and disparities in dataset sizes and measurement to test this hypothesis. |
| Outcome: | The proposed model shows that fusional languages perform better than fusionative languages . the authors suggest that morphological typology may explain some of the performance differences . |
Weight Tying Biases Token Embeddings Towards the Output Space (2026.findings-acl)
Copied to clipboard
| Challenge: | Weight tying is a common practice in language model design, but its impact on learning embedding space remains unclear. |
| Approach: | They show that weight tying optimizes the embedding matrix for output prediction . they also show that tied embeddable matrices align more closely with output embedders . |
| Outcome: | The proposed weight tying approach harms performance at scale and has implications for training smaller LLMs. |
BPE Gets Picky: Efficient Vocabulary Refinement During Tokenizer Training (2024.emnlp-main)
Copied to clipboard
| Challenge: | Tokenization is a relatively understudied area, but it can greatly impact model performance and efficiency. |
| Approach: | They propose a modified BPE tokenizer that removes merges that leave intermediate "junk" tokens from the vocabulary. |
| Outcome: | The proposed method improves vocabulary efficiency, eliminates under-trained tokens, and does not compromise text compression. |