Papers by Marius Mosbach
CLaS-Bench: A Cross-Lingual Alignment and Steering Benchmark (2026.findings-acl)
Copied to clipboard
Daniil Gurgurov, Yusser Al Ghussin, Tanja Baeumel, Cheng-Ting Chou, Patrick Schramowski, Marius Mosbach, Josef Van Genabith, Simon Ostermann
| Challenge: | Understanding and controlling behavior of large language models (LLMs) is an important topic in multilingual NLP. |
| Approach: | They propose a lightweight parallel-question benchmark for evaluating language-forcing behavior in large language models across 32 languages. |
| Outcome: | The proposed benchmark measures language steering in 32 languages across 32 languages. |
From Insights to Actions: The Impact of Interpretability and Analysis Research on NLP (2024.emnlp-main)
Copied to clipboard
| Challenge: | Interpretability and analysis (IA) research is a growing subfield within NLP . a criticism of this work is that it lacks actionable insights and therefore has little impact on NLP. |
| Approach: | They propose to quantify the impact of interpretation and analysis research on NLP . they use citation graphs and a survey to find out what is missing in IA research . |
| Outcome: | The proposed study shows that IA research is well-cited outside of IA and central in the NLP citation graph. |
Exploring the Effectiveness and Consistency of Task Selection in Intermediate-Task Transfer Learning (2024.acl-srw)
Copied to clipboard
| Challenge: | Identifying beneficial tasks to transfer from is a critical step toward successful intermediate-task transfer learning. |
| Approach: | They propose a method that measures pairwise token similarity using maximum inner product search to improve task prediction. |
| Outcome: | The proposed method improves task prediction scores from 2.59% to 3.96% for tasks requiring reasoning abilities, but not for reasoning abilities. |
Weaker Than You Think: A Critical Look at Weakly Supervised Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Weakly supervised learning is a popular approach for training machine learning models in low-resource settings. |
| Approach: | They propose to use weakly supervised learning to train models with noisy labels from weak sources instead of collecting expensive human annotations. |
| Outcome: | The proposed methods outperform weakly supervised methods on various NLP datasets and tasks on the test sets. |
Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuning (2022.coling-1)
Copied to clipboard
| Challenge: | Multilingual pre-trained language models have shown impressive performance on several downstream tasks for both high-resourced and low-resource languages. |
| Approach: | They propose to apply multilingual adaptive fine-tuning to 17 most-resourced African languages and three other high-resource languages to encourage cross-lingual transfer learning. |
| Outcome: | The proposed approach is competitive to LAFT on individual languages while requiring significantly less disk space. |
Do Generalisation Results Generalise? (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies evaluating generalisation performance on large language models focuses on a single out-of-distribution dataset . |
| Approach: | They examine whether OOD generalisation results generalise across multiple OOD testsets throughout a finetuning run and then evaluate the partial correlation of results . |
| Outcome: | The proposed model achieves high scores on multiple OOD testsets, regressing out in-domain performance. |
Understanding the Influence of Synthetic Data for Text Embedders (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in general purpose text embedders have been driven by training on synthetic training data. |
| Approach: | They propose to use GPT-4 to produce high quality synthetic data that expands existing training datasets for embeddings to new tasks. |
| Outcome: | The proposed dataset is high quality and leads to consistent improvements in performance. |
The Impact of Demonstrations on Multilingual In-Context Learning: A Multidimensional Analysis (2024.findings-acl)
Copied to clipboard
Miaoran Zhang, Vagrant Gautam, Mingyang Wang, Jesujoba Alabi, Xiaoyu Shen, Dietrich Klakow, Marius Mosbach
| Challenge: | In-context learning is a popular inference strategy where large language models solve a task using only a few labeled demonstrations without updating the model parameters. |
| Approach: | They conduct multidimensional analysis of multilingual in-context learning using 5 models from different model families and 9 datasets covering classification and generation tasks. |
| Outcome: | The results show that demonstrations vary significantly across models, tasks, and languages. |
On the Interplay Between Fine-tuning and Sentence-level Probing for Linguistic Knowledge in Pre-trained Transformers (2020.findings-emnlp)
Copied to clipboard
| Challenge: | linguistic knowledge encoded in pre-trained contextual embeddings is poorly understood . fine-tuning can be used to investigate the representations of pre-train models . |
| Approach: | They propose to investigate fine-tuning of contextualized embedding models through sentence-level probing. |
| Outcome: | The proposed method improves probing accuracy for three pre-trained models. |
MCSE: Multimodal Contrastive Learning of Sentence Embeddings (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to learning semantically meaningful sentence embeddings are limited by the complexity of pre-trained models. |
| Approach: | They propose a sentence embedding learning approach that exploits both visual and textual information via a multimodal contrastive objective. |
| Outcome: | The proposed approach improves the state-of-the-art average Spearman’s correlation by 1.7% on a variety of semantic textual similarity tasks. |
A Closer Look at Linguistic Knowledge in Masked Language Models: The Case of Relative Clauses in American English (2020.coling-main)
Copied to clipboard
Marius Mosbach, Stefania Degaetano-Ortlieb, Marie-Pauline Krielke, Badr M. Abdullah, Dietrich Klakow
| Challenge: | Despite the high performance of transformer-based language models, we still lack understanding of the kind of linguistic knowledge they learn and rely on. |
| Approach: | They evaluate three transformer-based language models and test their grammatical and semantic knowledge by sentence-level probing, diagnostic cases, and masked prediction tasks. |
| Outcome: | The models capture grammatical and semantic knowledge, but they lack model-specific weaknesses especially on semantic knowledge. |
Few-shot Fine-tuning vs. In-context Learning: A Fair Comparison and Evaluation (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies show that in-context learning and few-shot fine-tuning can generalize well out-of-domain. |
| Approach: | They compare few-shot fine-tuning and in-context learning for task adaptation . they find that both approaches generalize similarly, but exhibit large variation . |
| Outcome: | The proposed methods outperform in-context learning and few-shot fine-tuning with OPT models of different sizes. |
The Hidden Space of Transformer Language Adapters (2024.acl-long)
Copied to clipboard
| Challenge: | Adapters are small modules trained on top of a frozen language model to adapt predictions to new target languages. |
| Approach: | They propose to train transformer language adapters on top of a frozen model to adapt predictions to new target languages. |
| Outcome: | The transformer language adapters are trained on top of a frozen model to adapt predictions to new target languages. |