Papers by Max Müller-Eberstein
Subspace Chronicles: How Linguistic Information Emerges, Shifts and Interacts during Language Model Training (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Contemporary advances in NLP are built on the representational power of latent embedding spaces learned by self-supervised language models (LMs). |
| Approach: | They use a new information theoretic probing suite to analyze representational subspaces in language models. |
| Outcome: | The proposed approach compared performance of nine tasks across 2M pre-training steps and five seeds. |
DECAF: A Dynamically Extensible Corpus Analysis Framework (2025.acl-demo)
Copied to clipboard
| Challenge: | DeCAF is an open-source Python library that enables the analysis and filtering of linguistically-annotated datasets down to the character level. |
| Approach: | They propose a framework that enables the analysis and filtering of linguistically-annotated datasets down to the character level. |
| Outcome: | The proposed framework analyzes a parsed version of the 115M-word BabyLM corpus and generates highly controlled and reproducible experimental settings targeting specific research questions. |
Establishing Trustworthiness: Rethinking Tasks and Model Evaluation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Language understanding is a multi-faceted cognitive capability, which the Natural Language Processing community has striven to model computationally for decades. |
| Approach: | They propose to rethink what constitutes tasks and model evaluation in NLP and pursue a more holistic view on language, placing trustworthiness at the center. |
| Outcome: | The proposed models are based on generative models and are being deployed in more real-world scenarios, including previously unforeseen zero-shot setups. |
Sort by Structure: Language Model Ranking as Dependency Probing (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing algorithms for pre-trained language models lack performance indicators for linguistic tasks such as structured prediction. |
| Approach: | They propose to measure the degree to which labeled trees are recoverable from an LM’s contextualized embeddings by probing to rank LMs for parsing dependencies in a given language. |
| Outcome: | The proposed approach predicts the best LM choice 79% of the time using less compute than training a full parser. |
Spectral Probing (2022.emnlp-main)
Copied to clipboard
| Challenge: | Contextualized embeddings capture linguistic phenomena at distinct layers and frequencies . current studies rely on handcrafted filters and are limited to English . |
| Approach: | They develop a fully learnable frequency filter to identify spectral profiles for any given task. |
| Outcome: | The proposed filter enables more granular analyses than handcrafted filters and improves efficiency. |
Experimental Standards for Deep Learning in Natural Language Processing Research (2022.findings-emnlp)
Copied to clipboard
Dennis Ulmer, Elisa Bassignana, Max Müller-Eberstein, Daniel Varab, Mike Zhang, Rob van der Goot, Christian Hardmeier, Barbara Plank
| Challenge: | a lack of common experimental standards remains an open challenge to the field at large . |
| Approach: | They propose to distill discussions on experimental standards into a single, widely-applicable methodology. |
| Outcome: | Using best practices, we can strengthen experimental evidence, improve reproducibility and enable scientific progress. |
Can Humans Identify Domains? (2024.lrec-main)
Copied to clipboard
Maria Barrett, Max Müller-Eberstein, Elisa Bassignana, Amalie Brogaard Pauli, Mike Zhang, Rob van der Goot
| Challenge: | Textual domain is a crucial property within the Natural Language Processing community due to its effects on downstream model performance. |
| Approach: | They examine the level of human disagreement and the relative difficulty of each annotation task by training classifiers to perform the same task. |
| Outcome: | The authors show that human proficiency in identifying related intrinsic textual properties is low and that disagreements are high. |
Genre as Weak Supervision for Cross-lingual Dependency Parsing (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that monolingual masked language models learn to represent data-driven notions of language variation. |
| Approach: | They harness genre metadata as a weak supervision signal for targeted data selection in zero-shot dependency parsing. |
| Outcome: | The proposed method outperforms baseline and embedding-based methods for 12 low-resource language treebanks and three of these target languages. |
When Meanings Meet: Investigating the Emergence and Quality of Shared Concept Spaces during Multilingual Language Model Training (2026.eacl-long)
Copied to clipboard
| Challenge: | Recent studies have found that Large Language Models process multilingual inputs in shared concept spaces, thought to support generalization and cross-lingual transfer. |
| Approach: | They investigate the development of language-agnostic concept spaces during pretraining of EuroLLM using the causal interpretability method of activation patching. |
| Outcome: | The proposed model is language-agnostic and enables cross-lingual transfer . the model is able to process multilingual inputs, but lacks cross-linguistic alignment . |
Analysing Differences in Persuasive Language in LLM-Generated Text: Uncovering Stereotypical Gender Patterns (2026.findings-acl)
Copied to clipboard
| Challenge: | Prior work has shown that large language models can successfully persuade humans and amplify persuasive language. |
| Approach: | They propose a framework for evaluating how persuasive language generation is affected by recipient gender, sender intent, or output language. |
| Outcome: | The proposed framework varies persuasive language when the recipient gender is specified or when the sender intent is specified. |
Probing for Labeled Dependency Trees (2022.acl-long)
Copied to clipboard
| Challenge: | Existing linear probes are limited to extracting undirected or unlabeled dependency parse trees, thereby capturing only a subset of the full task. |
| Approach: | They propose a linear probe which can extract labeled and directed dependency parse trees from embeddings while using fewer parameters and compute than prior methods. |
| Outcome: | The proposed method outperforms baselines and prior work in 13 languages and outperformed competitive baselines. |
Frustratingly Easy Performance Improvements for Low-resource Setups: A Tale on BERT and Segment Embeddings (2022.lrec-1)
Copied to clipboard
| Challenge: | Understanding why contextualized embeddings work is still an active area of research. |
| Approach: | They propose to use a BERT architecture to encode a sub-word, position and a segment embedding as input representations for each sub- word. |
| Outcome: | The proposed model performs well on single-sentence prediction tasks while swapping segment IDs in paired-sentent tasks. |
Enough Is Enough! a Case Study on the Effect of Data Size for Evaluation Using Universal Dependencies (2024.lrec-main)
Copied to clipboard
| Challenge: | a recent study suggests that large datasets require large amounts of simulated data and scores to perform model evaluation. |
| Approach: | They propose to use dependency parsing to optimize annotation efforts by using a sample size of 5,000 tokens for in-language in-domain datasets. |
| Outcome: | The proposed method can be applied to other tasks, including classification and translation tasks. |
Evidence > Intuition: Transferability Estimation for Encoder Selection (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on LM transferability have focused on a priori tuning of encoders . prior work has examined the different yet related tasks of performance prediction . |
| Approach: | They propose to generate quantitative evidence to predict which LM will perform best on a target task without fine-tuning all candidates. |
| Outcome: | The proposed model outperforms the standard of human practitioner ranking in 94% of the setups. |