Papers by Rochelle Choenni
How do languages influence each other? Studying cross-lingual data sharing during LM fine-tuning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual language models can learn generalisations useful for other languages . yet, it remains unclear to what extent and under which conditions these models benefit from multilingual data and cross-lingual sharing. |
| Approach: | They propose a training data attribution method to retrieve training samples from multilingual data that are most influential for test predictions in a given language. |
| Outcome: | The proposed method exploits the ability to learn generalisations useful for other languages on zero-shot cross-lingual transfer for many languages. |
Best-of-L: Cross-Lingual Reward Modeling for Mathematical Reasoning (2026.findings-eacl)
Copied to clipboard
| Challenge: | Recent studies have focused on improving reasoning ability in English models, with multilingual models receiving comparatively little attention. |
| Approach: | They propose a framework that ranks candidate reasoning traces across languages rather than within a single language. |
| Outcome: | The proposed framework improves accuracy by up to 10 points in English compared to using reward modeling within a single language. |
Finding Culture-Sensitive Neurons in Vision-Language Models (2026.eacl-long)
Copied to clipboard
| Challenge: | Vision-language models struggle on culturally situated inputs, study shows . despite impressive performance, many VLMs struggle on such culturally grounded inputs . |
| Approach: | They propose a new margin-based selector to identify neurons associated with cultural selectivity . they also introduce a model-dependent decoder to identify such neurons . |
| Outcome: | The proposed model outperforms probability- and entropy-based methods in identifying neurons associated with cultural selectivity. |
Examining Modularity in Multilingual LMs via Language-Specialized Subnetworks (2024.findings-naacl)
Copied to clipboard
| Challenge: | Recent work has proposed explicitly inducing language-wise modularity in multilingual LMs via sparse fine-tuning (SFT) on per-language subnetworks as a means of better guiding cross-lingual sharing. |
| Approach: | They propose to explicitly inducing language-wise modularity in multilingual LMs via sparse fine-tuning on per-language subnetworks to better guide cross-lingual sharing. |
| Outcome: | The proposed approach can increase language specialization of subnetworks in favor of more cross-lingual sharing. |
Probing LLMs for Joint Encoding of Linguistic Categories (2023.findings-emnlp)
Copied to clipboard
Giulio Starace, Konstantinos Papakostas, Rochelle Choenni, Apostolos Panagiotopoulos, Matteo Rosati, Alina Leidinger, Ekaterina Shutova
| Challenge: | Existing research suggests that a linguistic hierarchy emerges in large language models . little is known about how encodings of different linguistic phenomena interact within the models - and to what extent processing of linguistically-related categories relies on the same, shared model representations. |
| Approach: | They propose a framework for testing the joint encoding of linguistic categories in large language models. |
| Outcome: | The proposed framework shows that the same patterns hold across languages in multilingual LLMs. |
Stepmothers are mean and academics are pretentious: What do pretrained language models learn about you? (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on "gender bias" and "racial bias" focus on stereotypical attributes of word representations . a new method to elicit stereotypical information is proposed to capture stereotypical traits in language models . |
| Approach: | They propose a method to elicit stereotypical information from pretrained language models . they use fine-tuning on news sources to study their emotional effects . |
| Outcome: | The proposed method can be used to analyze emotion and stereotype shifts due to linguistic experience using fine-tuning on news sources. |
Local Contrastive Editing of Gender Stereotypes (2024.emnlp-main)
Copied to clipboard
| Challenge: | Stereotypical bias encoded in language models (LMs) poses a threat to safe language technology . current research lacks a thorough understanding of manifestations of biases in specific model weights. |
| Approach: | They propose a method that localizes and edits weights associated with gender bias . they use local contrastive editing to localize and control a small subset of weights . |
| Outcome: | The proposed method localizes and controls a small subset of weights that encode gender bias. |
Metaphor Understanding Challenge Dataset for LLMs (2024.acl-long)
Copied to clipboard
| Challenge: | Metaphor understanding is an essential task for large language models (LLMs). |
| Approach: | They propose to evaluate the metaphor understanding capabilities of large language models (LLMs) the metaphor understanding challenge dataset provides over 10k paraphrases and 1.5k instances of inapt paraphrase. |
| Outcome: | The metaphor understanding challenge dataset evaluates the performance of large language models on a range of NLU tasks. |
M-Wanda: Improving One-Shot Pruning for Multilingual LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual LLM performance is often dependent on model size, resulting in performance loss. |
| Approach: | They propose a pruning method that models cross-lingual variation by incorporating language-aware activation statistics into its pruning criterion and dynamically adjusts layerwise sparsity based on cross-linguistic importance. |
| Outcome: | The proposed method improves performance at minimal additional costs while maintaining multilinguality. |