Papers by Benjamin Bergen
Correlations between Multilingual Language Model Geometry and Crosslingual Transfer Performance (2024.lrec-main)
Copied to clipboard
| Challenge: | Pre-trained multilingual language models represent multiple languages in a single vector space, a feature hypothesized to enable impressive crosslingual transfer capabilities. |
| Approach: | They propose to use a multilingual representation space that sorts axes based on their language-separability to determine whether geometric distances between languages correlate with crosslingual transfer performance. |
| Outcome: | The proposed measures do not generalize well across models, layers, and tasks. |
Rarely a problem? Language models exhibit inverse scaling in their predictions following few-type quantifiers (2023.findings-acl)
Copied to clipboard
| Challenge: | Current work suggests that language models deal poorly with quantifiers-they struggle to predict which quantifier is used in a given context and also perform poorly at generating appropriate continuations following logical quantifier. |
| Approach: | They propose to use 960 English sentence stimuli to build 22 autoregressive transformer models of different sizes to test their performance on ‘few’-type quantifiers. |
| Outcome: | The proposed models perform poorly on ‘few’-type quantifiers, and the larger the model, the worse its performance. |
Why do language models perform worse for morphologically complex languages? (2025.coling-main)
Copied to clipboard
| Challenge: | Language models perform differently across languages, a new study suggests . morphological typology may explain some of the performance differences, authors say . |
| Approach: | They propose to test morphological alignment of tokenizers, tokenization quality and disparities in dataset sizes and measurement to test this hypothesis. |
| Outcome: | The proposed model shows that fusional languages perform better than fusionative languages . the authors suggest that morphological typology may explain some of the performance differences . |
RAW-C: Relatedness of Ambiguous Words in Context (A New Lexical Resource for English) (2021.acl-long)
Copied to clipboard
| Challenge: | lexical ambiguity is a problem for NLP, but few tasks evaluate its impact on human intuitions. |
| Approach: | They propose to use contextualized word embeddings to evaluate word meaning . they use a dataset of human relatedness judgments and human estimates of sense dominance . |
| Outcome: | The proposed model matches human intuitions with contextualized embeddings on 112 ambiguous words in context with 672 sentence pairs. |
The Geometry of Multilingual Language Model Representations (2022.emnlp-main)
Copied to clipboard
| Challenge: | XLM-R models encode language-sensitive information in each language, allowing them to extract features for downstream tasks and cross-lingual transfer learning. |
| Approach: | They evaluate how multilingual language models maintain a shared multilingual representation space while still encoding language-sensitive information in each language. |
| Outcome: | The proposed model can extract features for downstream tasks and cross-lingual transfer learning. |