Papers by Isabel Papadimitriou
Investigating the interaction of linguistic and mathematical reasoning in language models using multilingual number puzzles (2025.emnlp-main)
Copied to clipboard
| Challenge: | Across languages, numeral systems vary widely in how they construct and combine numbers. |
| Approach: | They conduct experiments to examine the linguistic and mathematical aspects of numbers in language. |
| Outcome: | The models can't solve linguistic-mathematical puzzles involving cross-linguistic numeral systems, the authors found . they lack the ability to flexibly infer compositional rules from implicit patterns in human-scale data. |
Using Shapley interactions to understand how models use structure (2025.acl-long)
Copied to clipboard
| Challenge: | Language models are intricately structured systems, and attribution measures are important for understanding how they combine features to influence outputs. |
| Approach: | They use Shapley Taylor interaction indices to examine how language and speech models internally relate and structure their inputs. |
| Outcome: | The proposed methods show that language models encode phonetic interactions . they show that the inputs are more entangled for pairs where a consonant influences a vowel or approximant . |
Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets (2022.tacl-1)
Copied to clipboard
Julia Kreutzer, Isaac Caswell, Lisa Wang, Ahsan Wahab, Daan van Esch, Nasanbayar Ulzii-Orshikh, Allahsera Tapo, Nishant Subramani, Artem Sokolov, Claytone Sikasote, Monang Setyawan, Supheakmungkol Sarin, Sokhar Samb, Benoît Sagot, Clara Rivera, Annette Rios, Isabel Papadimitriou, Salomey Osei, Pedro Ortiz Suarez, Iroro Orife, Kelechi Ogueji, Andre Niyongabo Rubungo, Toan Q. Nguyen, Mathias Müller, André Müller, Shamsuddeen Hassan Muhammad, Nanda Muhammad, Ayanda Mnyakeni, Jamshidbek Mirzakhalov, Tapiwanashe Matangira, Colin Leong, Nze Lawson, Sneha Kudugunta, Yacine Jernite, Mathias Jenny, Orhan Firat, Bonaventure F. P. Dossou, Sakhile Dlamini, Nisansa de Silva, Sakine Çabuk Ballı, Stella Biderman, Alessia Battisti, Ahmed Baruwa, Ankur Bapna, Pallavi Baljekar, Israel Abebe Azime, Ayodele Awokoya, Duygu Ataman, Orevaoghene Ahia, Oghenefego Ahia, Sweta Agrawal, Mofetoluwa Adeyemi
| Challenge: | Lower-resource corpora have systematic issues, including mislabeled or nonstandard/ambiguous language codes. |
| Approach: | They manually audit the quality of 205 language-specific corpora released with five major public datasets. |
| Outcome: | The results show that lower-resource corpora have systematic issues even for non-proficient speakers. |
Learning Music Helps You Read: Using Transfer to Study Linguistic Structure in Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | et al., 2018a, 2018b) show that LSTMs can transfer from non-linguistic data to natural language models with different types of abstract structure. |
| Approach: | They propose to use transfer learning to analyze encoding of grammatical structure in neural language models. |
| Outcome: | The proposed method improves test performance on natural language despite no overlap in surface form or vocabulary. |
Multilingual BERT has an accent: Evaluating English influences on fluency in multilingual models (2023.findings-eacl)
Copied to clipboard
| Challenge: | Multilingual models can improve NLP performance on low-resource languages by leveraging higher-resourced languages, but they also reduce average performance on all languages. |
| Approach: | They propose a method to evaluate multilingual models by asking if models predict languages with an 'English accent' they propose to use grammatical structure bias to determine if multilingual model is biased toward English-like setting . |
| Outcome: | The proposed method compares the fluency of multilingual models to the fluencies of monolingual Spanish and Greek models. |
Deep Subjecthood: Higher-Order Grammatical Features in Multilingual BERT (2021.eacl-main)
Copied to clipboard
| Challenge: | a recent study has shown that multilingual BERT encodes sentences in structurally meaningful ways. |
| Approach: | They analyze how morphosyntactic alignment manifests across embedding spaces of languages . they train classifiers to recover subjecthood of mBERT embedds in transitive sentences . |
| Outcome: | The proposed model encodes a high-order grammatical feature of morphosyntactic alignment across languages . the results show that the classifier distributions reflect the morphological alignment of their training languages based on the results . |
Oolong: Investigating What Makes Transfer Learning Hard with Controlled Studies (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) require vast datasets for pretraining, making it difficult to train LLMs from scratch for lowresource languages. |
| Approach: | They propose to transform a language of the GLUE benchmark and then fine tune a pretrained model on that dataset. |
| Outcome: | The proposed models recover from syntactic-style shifts, but cannot recover from vocabulary misalignment and embedding matrix re-initialization, even with continued pretraining on 15 million tokens. |
When classifying grammatical role, BERT doesn’t care about word order... except when it matters (2022.acl-short)
Copied to clipboard
| Challenge: | Recent work has shown large language models are surprisingly word order invariant . however, word order knowledge is crucial in defining later-layer representations of words . |
| Approach: | They probe grammatical role representations in English BERT and GPT-2 to find word order crucial . they find word orders are crucial in defining later-layer representations of words in non-prototypical positions . |
| Outcome: | The proposed model is based on natural prototypical inputs where word order is crucial for correct classification. |
Injecting structural hints: Using language models to study inductive biases in language learning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study examines the cognitive inductive biases that make language learning possible. |
| Approach: | They structurally bias transformer language models by pretraining on synthetic data . they then evaluate their inductive biases by fine-tuning on three different languages . |
| Outcome: | The proposed method predisposes transformer models to three types of inductive biases . it also fine-tunes the models on three typologically-distant human languages . |