Papers by Hinrich Schuetze

LMTurk: Few-Shot Learners as Crowdsourcing Workers in a Language-Model-as-a-Service Framework (2022.findings-naacl)

Challenge:	Existing methods to overcome catastrophic forgetting are rehearsal-based and parameter isolation-based.
Approach:	They propose a rehearsal-free framework which continuously adds new modules to language models and composes them with existing modules.
Outcome:	Experiments on benchmarks show that MoCL outperforms state-of-the-art and effectively facilitates knowledge transfer.

Copied to clipboard

Mengjie Zhao, Fei Mi, Yasheng Wang, Minglei Li, Xin Jiang, Qun Liu, Hinrich Schuetze

Modeling Ideological Salience and Framing in Polarized Online Groups with Graph Neural Networks and Structured Sparsity (2022.findings-naacl)

Challenge:	Recent work shows that large-scale pretrained language models (PLMs) are effective few-shot learners.
Approach:	They propose a method that treats few-shotlearners as crowdsourcing workers . they propose to use these workers to train models that solve a task well .
Outcome:	The proposed approach treats few-shotlearners as crowdsourcing workers . the resulting annotations can be utilized to train models that solve the task well .

Copied to clipboard

Valentin Hofmann, Xiaowen Dong, Janet Pierrehumbert, Hinrich Schuetze

Multimodality

Challenge:	Existing methods to detect ideological divides in social media rely on knowing in advance the political orientation of text . fascist and mainstream are among the most polarized concepts in reddit in 2019 .
Approach:	They propose a minimally supervised method that leverages the network structure of online discussion forums to detect polarized concepts.
Outcome:	The proposed framework captures temporal ideological dynamics such as right-wing and left-wing radicalization using graph neural networks and sparsity learning.

Modular and Parameter-Efficient Multimodal Fusion with Prompting (2022.findings-acl)

Copied to clipboard

Sheng Liang, Mengjie Zhao, Hinrich Schuetze

Self-Evolving Multi-Agent Systems via Textual Backpropagation (2026.findings-acl)

Challenge:	Recent research has made impressive progress in large-scale multimodal pre-training.
Approach:	They propose to use prompt vectors to align multimodal modalities by pretraining text inputs with prompts or embedding vectors.
Outcome:	The proposed method achieves comparable performance to several other multimodal fusion methods in low-resource settings.

Copied to clipboard

Xiaowen Ma, Yunpu Ma, Chenyang Lin, Sikuan Yan, Jinhe Bi, Zixuan Cao, Yijun Tian, Volker Tresp, Hinrich Schuetze

Reasoning

Challenge:	Large Language Models (LLMs) have proven effective for addressing complex, high-dimensional tasks, but current approaches rely on static, manually engineered multi-agent configurations.
Approach:	They propose a framework that conceptualizes multi-agent collaboration as a layered neural network architecture.
Outcome:	The proposed framework surpasses leading multi-agent baselines under the same configurations, showing consistent performance improvements.

Better Call SAUL: Fluent and Consistent Language Model Editing with Generation Regularization (2024.findings-emnlp)

Copied to clipboard

Mingyang Wang, Lukas Lange, Heike Adel, Jannik Strötgen, Hinrich Schuetze

GNNavi: Navigating the Information Flow in Large Language Models by Graph Neural Network (2024.findings-acl)

Challenge:	State-of-the-art methods for updating large language models require computational overhead and lack theoretical validation.
Approach:	They propose a model editing method that uses sentence concatenation with augmented random facts for generation regularization.
Outcome:	The proposed method outperforms state-of-the-art methods while maintaining generation quality and reducing computational overhead.

Copied to clipboard

Shuzhou Yuan, Ercong Nie, Michael Färber, Helmut Schmid, Hinrich Schuetze

Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning (2026.acl-long)

Challenge:	Large Language Models (LLMs) exhibit strong In-Context Learning (ICL) capabilities when prompts with demonstrations are used.
Approach:	They propose a prompt-based parameter-efficient fine-tuning approach that leverages insights into ICL’s information flow dynamics and hardwires the desired information flow into the GNN.
Outcome:	The proposed approach surpasses prompt-based fine-tuning methods in few-shot settings by updating just 0.2% to 0.5% of parameters.

Copied to clipboard

Sikuan Yan, Xiufeng Yang, Zuchao Huang, Ercong Nie, Zifeng Ding, Zonggen Li, Xiaowen Ma, Jinhe Bi, Kristian Kersting, Jeff Z. Pan, Hinrich Schuetze, Volker Tresp, Yunpu Ma

Lost in Multilinguality: Dissecting Cross-lingual Factual Inconsistency in Transformer Language Models (2025.acl-long)

Challenge:	Large Language Models (LLMs) are stateless and limited by a finite context window, preventing them from maintaining knowledge across long conversations or evolving tasks.
Approach:	They propose a reinforcement learning framework that empowers LLMs to actively manage external memory through two specialized agents.
Outcome:	The proposed framework outperforms baselines and benchmarks across diverse question types, three benchmarks, and multiple model scales.

Copied to clipboard

Mingyang Wang, Heike Adel, Lukas Lange, Yihong Liu, Ercong Nie, Jannik Strötgen, Hinrich Schuetze

Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models (2026.findings-acl)

Challenge:	Multilingual language models store factual knowledge across languages but struggle to provide consistent responses to semantically equivalent prompts in different languages.
Approach:	They propose a linear shortcut method that bypasses computations in the final layers . this method improves accuracy and cross-lingual consistency .
Outcome:	The proposed method improves prediction accuracy and cross-lingual consistency.

Copied to clipboard

Hengyuan Zhang, Zhihao Zhang, Ercong Nie, Mingyang Wang, Zunhai Su, Yiwei Wang, Qianli Wang, Shuzhou Yuan, Xufeng Duan, Qibo Xue, Zeping Yu, Chenming Shang, Xiao Liang, Jing Xiong, Hui Shen, Chaofan Tao, Zhengwu Liu, Senjie Jin, Zhiheng Xi, Dongdong Zhang, Sophia Ananiadou, Tao Gui, Ruobing Xie, Hayden Kwok-Hay So, Hinrich Schuetze, Xuanjing Huang, Qi Zhang, Ngai Wong

Explainability & Interpretability in NLP

Challenge:	Existing literature on mechanistic interpretation (MI) treats it as an observational science, leaving practical applications underexplored.
Approach:	They propose a survey structured around the pipeline to identify and improve MI models.
Outcome:	The proposed framework enables tangible improvements in Alignment, Capability, and Efficiency.

Language-Agnostic Bias Detection in Language Models with Bias Probing (2023.findings-emnlp)

Copied to clipboard

Abdullatif Köksal, Omer Yalcin, Ahmet Akbiyik, M. Kilavuz, Anna Korhonen, Hinrich Schuetze

LongForm: Effective Instruction Tuning with Reverse Instructions (2024.findings-emnlp)

Challenge:	Pretrained language models (PLMs) contain strong social biases, which are difficult to quantify because current methods focusing on fill-the-mask objectives are sensitive to slight changes in input.
Approach:	They propose a bias probing technique called LABDet to evaluate social bias in pretrained language models with a language-agnostic method.
Outcome:	The proposed method “surfaces” nationality bias by training a classifier on top of a frozen PLM on non-nationality sentiment detection.

Copied to clipboard

Abdullatif Köksal, Timo Schick, Anna Korhonen, Hinrich Schuetze

Time Course MechInterp: Analyzing the Evolution of Components and Knowledge in Large Language Models (2025.findings-acl)

Challenge:	Prior work on instruction tuning relies on expensive human annotation and crowd-sourced datasets with alignment issues.
Approach:	They propose a method to generate instructions via LLMs from human-written corpus examples using reverse instructions.
Outcome:	The proposed method outperforms larger language models without instruction tuning on tasks such as story/recipe generation and long-form question answering.

Copied to clipboard

Ahmad Dawar Hakimi, Ali Modarressi, Philipp Wicke, Hinrich Schuetze

MaskLID: Code-Switching Language Identification through Iterative Masking (2024.acl-short)

Challenge:	Large language models acquire and store factual knowledge for interpretability, reliability, efficiency . prior work on factual recall focused on localizing knowledge within transformer parameters .
Approach:	They analyze the evolution of factual knowledge representation in a large language model by tracking its attention heads and feed forward networks over training.
Outcome:	The proposed model acquires and stores factual knowledge over time and is adaptively trained . the proposed model can be pruned, optimized, and transparent .

Copied to clipboard

Amir Hossein Kargaran, François Yvon, Hinrich Schuetze

Parallel Universes, Parallel Languages: A Comprehensive Study on LLM-based Multilingual Counterfactual Example Generation (2026.acl-long)

Challenge:	Sentence-level LIDs are classifiers trained on monolingual texts to provide single labels, typically using a softmax layer to turn scores into probabilities.
Approach:	They propose a simple yet effective code-switching language identification method that uses the LID itself to mask features associated with L1 and L2 in the next round.
Outcome:	The proposed method is based on two open-source LIDs based in the FastText architecture and does not require any external resources.

Copied to clipboard

Qianli Wang, Van Bach Nguyen, Yihong Liu, Fedor Splitt, Nils Feldhus, Christin Seifert, Hinrich Schuetze, Sebastian Möller, Vera Schmitt

Large Reasoning Models Are (Not Yet) Multilingual Latent Reasoners (2026.findings-acl)

Challenge:	Large language models excel at generating English counterfactuals but their effectiveness in generating multilingual counterfacts remains unclear.
Approach:	They conduct automatic evaluations on both directly generated and derived counterfactuals in six languages and find that cross-lingual perturbations follow common strategic principles.
Outcome:	The proposed models show that translation-based counterfactuals offer higher validity than their directly generated counterparts, but still fall short of matching the quality of the original English counterf actuals.

Copied to clipboard

Yihong Liu, Raoyuan Zhao, Hinrich Schuetze, Michael A. Hedderich

Reasoning

Challenge:	Recent work shows that large reasoning models arrive at the correct answer before completing textual reasoning steps, indicating the presence of latent reasoning.
Approach:	They conduct a systematic investigation of multilingual latent reasoning in large reasoning models across 11 languages.
Outcome:	The proposed model arrive at the correct answer before completing the reasoning steps, indicating the presence of latent reasoning.

GradSim: Gradient-Based Language Grouping for Effective Multilingual Training (2023.emnlp-main)

Copied to clipboard

Mingyang Wang, Heike Adel, Lukas Lange, Jannik Strötgen, Hinrich Schuetze

Why Do More Experts Fail? A Theoretical Analysis of Model Merging (2026.acl-long)

Challenge:	Existing studies show that not all languages positively influence each other . multilingual training can help in those cases by sharing knowledge across languages .
Approach:	They propose a gradient similarity-based language grouping method for multilingual training that is better correlated with cross-lingual model performance.
Outcome:	The proposed method leads to the largest performance gains on a multilingual dataset and is better correlated with cross-lingual model performance.

Copied to clipboard

Zijing Wang, Xingle Xu, YongKang Liu, Yiqun Zhang, Peiqin Lin, Shi Feng, Daling Wang, Xiaocui Yang, Hinrich Schuetze

Challenge:	Existing methods for model merging struggle to maintain performance gains as the number of merged models increases.
Approach:	They propose a Reparameterized Heavy-Tailed method to extend the merged model’s coverage and enhance performance.
Outcome:	The proposed method extends the merged model’s coverage and enhances performance on 19 benchmarks, including knowledge-intensive and general-purpose tasks.

Evaluating Robustness of Large Language Models Against Multilingual Typographical Errors (2026.acl-long)

Copied to clipboard

Raoyuan Zhao, Yihong Liu, Lena Altinger, Hinrich Schuetze, Michael A. Hedderich

How Programming Concepts and Neurons Are Shared in Code Language Models (2025.findings-acl)

Challenge:	Large language models (LLMs) are increasingly deployed in multilingual, real-world applications where user inputs introduce typographical errors.
Approach:	They propose a multilingual typo generation algorithm that simulates human-like errors based on language-specific keyboard layouts and typing behavior.
Outcome:	The proposed model can generate the correct answer ("500") under typos in English, German, and Russian.

Copied to clipboard

Amir Hossein Kargaran, Yihong Liu, François Yvon, Hinrich Schuetze

Large Language Models as Neurolinguistic Subjects: Discrepancy between Performance and Competence (2025.findings-acl)

Challenge:	Several studies have focused on programming languages in a monolingual setting, but most focus on programming language models.
Approach:	They perform a few-shot translation task on 21 PL pairs using two Llama-based models and decode the embeddings of intermediate layers.
Outcome:	The proposed model assigns high probability to English tokens in the second half of the intermediate layers and language-specific neurons are concentrated in the bottom layers . the model's concept space is closer to English (including PL keywords) and the model is more efficient at identifying language-related neurons.

Copied to clipboard

Linyang He, Ercong Nie, Helmut Schmid, Hinrich Schuetze, Nima Mesgarani, Jonathan Brennan

Kardeş-NLU: Transfer to Low-Resource Languages with Big Brother’s Help – A Benchmark and Evaluation for Turkic Languages (2024.eacl-long)

Challenge:	Existing evaluations of Large Language Models (LLMs) reflect statistical rules that may not accurately represent LLMs’ true linguistic competence.
Approach:	They propose a method that combines minimal pair and diagnostic probing to analyze activation patterns across model layers.
Outcome:	The proposed method combines minimal pair and diagnostic probing to analyze activation patterns across model layers.

Copied to clipboard

Lütfi Kerem Senel, Benedikt Ebing, Konul Baghirova, Hinrich Schuetze, Goran Glavaš

Your Pretrained Model Tells the Difficulty Itself: A Self-Adaptive Curriculum Learning Paradigm for Natural Language Understanding (2025.acl-srw)

Challenge:	Cross-lingual transfer (XLT) driven by massively multilingual language models (mmLMs) has been shown to be ineffective for low-resource (LR) target languages with little (or no) representation in mmLM’s pretraining .
Approach:	They propose a benchmark to evaluate cross-lingual transfer (XLT) to LR languages that do have a close HR relative and a framework to integrate Turkish into XLT.
Outcome:	The proposed configuration is of practical relevance for more of the world’s languages: XLT to LR languages that do have a close HR relative.

Copied to clipboard

Qi Feng, Yihong Liu, Hinrich Schuetze

Flow-Adapter Architecture for Unsupervised Machine Translation (2022.acl-long)

Challenge:	Existing curriculum learning approaches rely on manually defined difficulty metrics which may not accurately reflect the model’s own perspective.
Approach:	They propose a self-adaptive curriculum learning paradigm that prioritizes fine-tuning examples based on difficulty scores predicted by pre-trained language models (PLMs) they evaluate four datasets covering binary and multi-class classification tasks.
Outcome:	The proposed model leads to faster convergence and improved performance compared to standard random sampling.

Copied to clipboard

Yihong Liu, Haris Jabbar, Hinrich Schuetze

Unleashing the Multilingual Encoder Potential: Boosting Zero-Shot Performance via Probability Calibration (2023.findings-emnlp)

Challenge:	Recent advances in deep learning have boosted the development of neural machine translation (NMT).
Approach:	They propose a flow-adapter architecture for unsupervised neural machine translation that leverages normalizing flows to model distributions of sentence-level latent representations.
Outcome:	The proposed model achieves competitive results on several unsupervised MT benchmarks.

Copied to clipboard

Ercong Nie, Helmut Schmid, Hinrich Schuetze

SynthEval: Hybrid Behavioral Testing of NLP Models with Synthetic Evaluation (2024.findings-emnlp)

Challenge:	Recent research demonstrates that multilingual encoder models are capable of zero-shot cross-lingual learning by using cloze-style prompts.
Approach:	They propose to reformulate input examples into cloze-style prompts to perform zero-shot multilingual tasks or linguistic probing by predicting label words at the masked token position.
Outcome:	The proposed method performs zero-shot multilingual tasks without updating parameters.

Copied to clipboard

Raoyuan Zhao, Abdullatif Köksal, Yihong Liu, Leonie Weissweiler, Anna Korhonen, Hinrich Schuetze

Challenge:	Existing frameworks for benchmarking in NLP often overestimate performance . however, manually creating a variety of test types requires significant human labor .
Approach:	They propose a framework that leverages large language models to generate a wide range of test types . they first generate sentences via LLMs and then identifies challenging examples .
Outcome:	The proposed framework overestimates performance on two classification tasks.

Collapse of Dense Retrievers: Short, Early, and Literal Biases Outranking Factual Evidence (2025.acl-long)

Copied to clipboard

Mohsen Fayyaz, Ali Modarressi, Hinrich Schuetze, Nanyun Peng

Information Retrieval

Challenge:	Notably, when multiple biases combine, models exhibit catastrophic performance degradation, selecting the answer-containing document in less than 10% of cases over a synthetic biased document without the answer.
Approach:	They repurpose a relation extraction dataset to quantify the impact of heuristic biases on retrievers like Dragon+ and Contriever.
Outcome:	The proposed models exhibit catastrophic performance degradation when multiple biases combine, selecting the answer-containing document in less than 10% of cases over a synthetic biased document without the answer.

mPLM-Sim: Better Cross-Lingual Similarity and Transfer in Multilingual Pretrained Language Models (2024.findings-eacl)

Copied to clipboard

Peiqin Lin, Chengzhi Hu, Zheyu Zhang, Andre Martins, Hinrich Schuetze

TurkishMMLU: Measuring Massive Multitask Language Understanding in Turkish (2024.findings-emnlp)

Challenge:	Recent multilingual pretrained language models encode strong language-specific signals, which are not explicitly provided during pretraining.
Approach:	They propose a language similarity measure that induces similarities across languages from mPLMs using multi-parallel corpora.
Outcome:	The proposed measure exhibits moderately high correlations with linguistic similarity measures, and more accurate similarity results on low correlation languages.

Copied to clipboard

Arda Yüksel, Abdullatif Köksal, Lütfi Kerem Senel, Anna Korhonen, Hinrich Schuetze

Reasoning

Challenge:	Existing multiple choice question answering benchmarks employ automatic translation for multilingual evaluation, but this approach is error-prone and potentially introduces culturally biased questions.
Approach:	They introduce the first multitask, multiple-choice Turkish QA benchmark, TurkishMMLU . they evaluate over 20 LLMs including open-source, closed-source and Turkish-adapted models .
Outcome:	The proposed benchmarks evaluate the reasoning, comprehension, and mathematical abilities of large language models.

Language Models with Rationality (2023.emnlp-main)

Copied to clipboard

Nora Kassner, Oyvind Tafjord, Ashish Sabharwal, Kyle Richardson, Hinrich Schuetze, Peter Clark

HiFT: A Hierarchical Full Parameter Fine-Tuning Strategy (2024.emnlp-main)

Challenge:	lack of interpretability is a growing impediment to widespread use of large language models . a new approach to solve this problem is to add a rational layer on top of the LLM .
Approach:	They propose to add a rational layer to the large language models to make model beliefs explicit . they also propose to identify and minimize contradictions in the model belief graph .
Outcome:	a new approach improves consistency without harming overall answer accuracy . the proposed approach makes model beliefs explicit and resolves inconsistencies .

Copied to clipboard

YongKang Liu, Yiqun Zhang, Qian Li, Tong Liu, Shi Feng, Daling Wang, Yifei Zhang, Hinrich Schuetze

An Embarrassingly Simple Method to Mitigate Undesirable Properties of Pretrained Language Model Tokenizers (2022.acl-short)

Challenge:	Existing approaches to fine-tuning language models use zeroth-order optimizers to conserve GPU memory.
Approach:	They propose a full-parameter fine-tuning strategy which updates a subset of parameters at each training step.
Outcome:	The proposed approach reduces the amount of gradients and optimizer state parameters residing in GPU memory at the same time, thereby reducing GPU memory usage.

Copied to clipboard

Valentin Hofmann, Hinrich Schuetze, Janet Pierrehumbert

Syntactic Text Processing

Challenge:	a standard tokenizer does not cover all characters of a word but preserves key aspects of its morphological structure . a novel method to improve tokenization of pretrained language models is proposed .
Approach:	They propose a method to improve the tokenization of pretrained language models . they use the vocabulary of a standard tokenizer but preserves morphological structure .
Outcome:	The proposed method improves tokenization of pretrained language models on morphological gold segmentations and text classification tasks.

OFA: A Framework of Initializing Unseen Subword Embeddings for Efficient Large-scale Multilingual Continued Pretraining (2024.findings-naacl)

Copied to clipboard

Yihong Liu, Peiqin Lin, Mingyang Wang, Hinrich Schuetze

SAD: A Large-Scale Strategic Argumentative Dialogue Dataset (2026.acl-long)

Challenge:	Existing methods to pretrain multilingual models are limited by the number of embedding parameters and the complexity of the model.
Approach:	They propose a framework that initializes the embeddings of unseen subwords and can adapt a model to multiple languages.
Outcome:	The proposed framework can adapt a pre-trained model to multiple languages efficiently and effectively.

Copied to clipboard

YongKang Liu, Jiayang Yu, Mingyang Wang, Yiqun Zhang, Ercong Nie, Shi Feng, Daling Wang, Kaisong Song, Hinrich Schuetze

Dialogue Systems & Conversational Agents

Challenge:	Argumentation is a key part of human reasoning and decision-making . existing argumentative corpora focus on single-turn settings, but multi-turn dialogues are often realized as multi-turned dialogues .
Approach:	They present a dataset for strategic multi-turn argumentation dialogues . they annotate each utterance with five strategy types, allowing multiple strategies per utterrance .
Outcome:	The proposed dataset shows that explicit prompting improves fluency, stylistic coherence and persuasiveness.

XAMPLER: Learning to Retrieve Cross-Lingual In-Context Examples (2025.findings-naacl)

Copied to clipboard

Peiqin Lin, Andre Martins, Hinrich Schuetze

Low-Resource NLP

Challenge:	XAMPLER: Cross-Lingual Example Retrieval is a cross-lingual example retrieval method . large language models (LLMs) have emerged as effective in-context learning methods .
Approach:	They propose a method to train a multilingual model with annotated English examples . they use annotized English data to train the model and use it to train other languages .
Outcome:	XAMPLER: Cross-Lingual Example Retrieval improves in-context learning in English . it trains a retriever based on a multilingual small language model using annotated English examples .

GlotLID: Language Identification for Low-Resource Languages (2023.findings-emnlp)

Copied to clipboard

Amir Kargaran, Ayyoob Imani, François Yvon, Hinrich Schuetze

XToM: Exploring the Multilingual Theory of Mind for Large Language Models (2026.acl-long)

Challenge:	Existing web-mined datasets for low-resource languages have been useful for low resource NLP.
Approach:	They propose a model that identifies 1665 low-resource languages and a new model that is rigorously evaluated and reliable.
Outcome:	The proposed model outperforms baselines when balancing F1 and false positive rate (FPR).

Copied to clipboard

Chunkit Chan, Yauwai Yim, Hongchuan Zeng, Zhiying Zou, Xinyuan Cheng, Zhifan Sun, Zheye Deng, Kawai Chung, Yuzhuo Ao, Fan Yixiang, Cheng Jiayang, Ercong Nie, Ginny Wong, Helmut Schmid, Hinrich Schuetze, Simon See, Yangqiu Song

MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment (2025.findings-acl)

Challenge:	Existing evaluations of ToM in LLMs are limited to English, neglecting the linguistic diversity that shapes human cognition.
Approach:	They propose a multilingual benchmark that evaluates ToM across five languages . they find that models excel in multilingual language understanding, but their ToM performance varies across languages.
Outcome:	The proposed benchmark evaluates LLMs across five languages and incorporates diverse task scenarios.

Copied to clipboard

Amir Hossein Kargaran, Ali Modarressi, Nafiseh Nikeghbal, Jana Diesner, François Yvon, Hinrich Schuetze

Consistent Document-level Relation Extraction via Counterfactuals (2024.findings-emnlp)

Challenge:	Existing benchmarks for multilinguality for English-centric large language models focus on classic tasks or cover a minimal number of languages.
Approach:	They propose a method to assess multilingual capabilities of pre-trained LLMs using parallel sentences.
Outcome:	The proposed method evaluates the multilingual capabilities of pre-trained English-centric models using parallel sentences.

Copied to clipboard

Ali Modarressi, Abdullatif Köksal, Hinrich Schuetze

A Comprehensive Evaluation of Multilingual Chain-of-Thought Reasoning: Performance, Consistency, and Faithfulness Across Languages (2026.findings-eacl)

Challenge:	Document-level relation extraction models trained on factual data exhibit inconsistent behavior, relying on spurious signals such as specific entities and external knowledge to extract triples.
Approach:	They propose a counterfactual data generation approach for document-level relation extraction datasets using entity replacement to generate triples from factual data.
Outcome:	The proposed approach extracts triples from factual data but fails on counterfactual modification.

Copied to clipboard

Raoyuan Zhao, Yihong Liu, Hinrich Schuetze, Michael A. Hedderich

Low-Resource NLP

Challenge:	Recent work has examined final-answer accuracy in multilingual settings, but the behavior of thinking traces, i.e., the intermediate steps that lead to the final answer, remains underexplored.
Approach:	They propose to measure language compliance, answer accuracy, and answer consistency when LRMs are explicitly instructed or prompt-hacked to think in a target language.
Outcome:	The proposed model improves in English and other high-resource languages while relying on traces to varying degrees.

Taxi1500: A Dataset for Multilingual Text Classification in 1500 Languages (2025.naacl-short)

Copied to clipboard

Chunlan Ma, Ayyoob Imani, Haotian Ye, Renhao Pei, Ehsaneddin Asgari, Hinrich Schuetze

BMIKE-53: Investigating Cross-Lingual Knowledge Editing with In-Context Learning (2025.acl-long)

Challenge:	a large-scale text classification dataset encompassing 1504 languages is needed to address this gap . low-resource languages are often overlooked due to the scarcity of evaluation datasets.
Approach:	They propose to use translations of the Bible to construct a large-scale text classification dataset that covers 1504 languages and annotate them using crowdsourcing.
Outcome:	The proposed dataset covers 1504 languages and is available to the public.

Copied to clipboard

Ercong Nie, Bo Shao, Mingyang Wang, Zifeng Ding, Helmut Schmid, Hinrich Schuetze

CoDA21: Evaluating Language Understanding Capabilities of NLP Models With Context-Definition Alignment (2022.acl-short)

Challenge:	Using a benchmark for cross-lingual knowledge editing, knowledge editing is underexplored.
Approach:	They propose a benchmark for cross-lingual in-context knowledge editing that spans 53 languages and three KE datasets.
Outcome:	The proposed benchmark systematically evaluates cross-lingual knowledge editing (IKE) under zero-shot, one-shot and few-shot setups.

Copied to clipboard

Lütfi Kerem Senel, Timo Schick, Hinrich Schuetze

Understanding In-Context Machine Translation for Low-Resource Languages: A Case Study on Manchu (2025.acl-long)

Challenge:	Pretrained language models (PLMs) have achieved superhuman performance on many benchmarks, creating a need for harder tasks.
Approach:	They propose a benchmark that measures natural language understanding (NLU) abilities of pretrained language models.
Outcome:	The proposed benchmark measures the ability of pretrained language models to perform on many tasks.

Copied to clipboard

Renhao Pei, Yihong Liu, Peiqin Lin, François Yvon, Hinrich Schuetze

Machine Translation

Challenge:	In-context machine translation (MT) with large language models can take advantage of linguistic resources such as grammar books and dictionaries.
Approach:	They propose to use in-context machine translation (MT) with large language models to take advantage of linguistic resources such as grammar books and dictionaries.
Outcome:	The proposed approach can take advantage of dictionaries and grammar books, but its performance is poor for many lowresource languages.

ImpliRet: Benchmarking the Implicit Fact Retrieval Challenge (2025.emnlp-main)

Copied to clipboard

Zeinab Sadat Taghavi, Ali Modarressi, Yunpu Ma, Hinrich Schuetze

Information Retrieval

Challenge:	Retrieval systems rely on surface-level cues such as keyword overlap and semantic similarity to evaluate retrieval beyond these shallow signals.
Approach:	They propose a benchmark that shifts the reasoning challenge to query-side processing techniques that can help resolve complexity.
Outcome:	The proposed benchmarks show that document-side reasoning remains a challenge.

MEAL: Stable and Active Learning for Few-Shot Prompting (2023.findings-emnlp)

Copied to clipboard

Abdullatif Köksal, Timo Schick, Hinrich Schuetze

Low-Resource NLP

Challenge:	Existing methods for few-shot classification have high variance across different sets of few shots and finetuning runs.
Approach:	They propose novel ensembling methods that significantly reduce run variability and introduce a new active learning criterion for data selection.
Outcome:	The proposed method significantly reduces run variability and improves performance on five tasks.

TransliCo: A Contrastive Learning Framework to Address the Script Barrier in Multilingual Pretrained Language Models (2024.acl-long)

Copied to clipboard

Yihong Liu, Chunlan Ma, Haotian Ye, Hinrich Schuetze

Mechanistic Understanding and Mitigation of Language Confusion in English-Centric Large Language Models (2025.findings-emnlp)

Challenge:	The world’s more than 7000 languages are written in at least 293 scripts, which poses a difficulty for multilingual pretrained language models in learning crosslingual knowledge through lexical overlap.
Approach:	They propose a framework that optimizes the Transliteration Contrastive Modeling objective to fine-tune an mPLM by contrasting sentences in its training data and transliterations in a unified script.
Outcome:	The proposed model outperforms Glot500-m on zero-shot crosslingual transfer tasks while retaining uniformity across scripts.

Copied to clipboard

Ercong Nie, Helmut Schmid, Hinrich Schuetze

On Relation-Specific Neurons in Large Language Models (2025.emnlp-main)

Challenge:	Language confusion is a critical challenge for large language models, especially for English-centric models.
Approach:	They propose to combine behavioral benchmarking with neuron-level analysis to study language confusion.
Outcome:	The proposed approach matches multilingual alignment in confusion reduction for many languages and yields cleaner, higher-quality outputs.

Copied to clipboard

Yihong Liu, Runsheng Chen, Lea Hirlimann, Ahmad Dawar Hakimi, Mingyang Wang, Amir Hossein Kargaran, Sascha Rothe, François Yvon, Hinrich Schuetze

Counting the Bugs in ChatGPT’s Wugs: A Multilingual Investigation into the Morphological Capabilities of a Large Language Model (2023.emnlp-main)

Challenge:	In large language models, certain neurons can store distinct pieces of knowledge learned during pretraining.
Approach:	They hypothesize that relation-specific neurons detect relation in input text and guide generation involving such a relation.
Outcome:	The proposed model can handle facts involving relation r and facts containing a different relation .

Copied to clipboard

Leonie Weissweiler, Valentin Hofmann, Anjali Kantharuban, Anna Cai, Ritam Dutt, Amey Hengle, Anubha Kabra, Atharva Kulkarni, Abhishek Vijayakumar, Haofei Yu, Hinrich Schuetze, Kemal Oflazer, David Mortensen

Morphology

Challenge:	Existing studies on large language models (LLMs) ignore the remarkable ability of humans to generalize and focus only on English.
Approach:	They conduct the first rigorous analysis of the morphological capabilities of ChatGPT in four typologically varied languages.
Outcome:	The proposed model massively underperforms purpose-built systems, particularly in English.

HYPEROFA: Expanding LLM Vocabulary to New Languages via Hypernetwork-Based Embedding Initialization (2025.acl-srw)

Copied to clipboard

Enes Özeren, Yihong Liu, Hinrich Schuetze

LangSAMP: Language-Script Aware Multilingual Pretraining (2025.acl-long)

Challenge:	Pre-trained language models exhibit suboptimal performance on mid- and low-resource languages due to limited exposure to these languages during pre-training.
Approach:	They propose a similarity-based subword embedding initialization heuristic that introduces new tokens specific to target languages, initializes their embedders, and applies continual pre-training on target-language data.
Outcome:	The proposed method outperforms random initialization baseline and matches or exceeds OFA in both continual pre-training convergence and downstream task performance.

Copied to clipboard

Yihong Liu, Haotian Ye, Chunlan Ma, Mingyang Wang, Hinrich Schuetze

Privacy-Preserving Federated Learning for Hate Speech Detection (2025.naacl-srw)

Challenge:	Recent multilingual pretrained language models often avoid using language embeddings, which places a significant burden on token representations to encode all language-specific information.
Approach:	They propose a method that incorporates both language and script embeddings into the output of Transformer blocks before passing the final representations to the language modeling head for prediction.
Outcome:	The proposed method outperforms the baseline model in zero-shot crosslingual transfer across diverse downstream tasks.

Copied to clipboard

Ivo De Souza Bueno Júnior, Haotian Ye, Axel Wisiorek, Hinrich Schuetze

GKnow: Measuring the Entanglement of Gender Bias and Factual Gender (2026.acl-long)

Challenge:	a federated learning system with differential privacy is tailored to low-resource languages . data with fewer than 20 sentences per client struggled due to excessive noise .
Approach:	They propose a federated learning system with differential privacy for hate speech detection . they fine-tuned pre-trained language models to find it to be the most effective .
Outcome:	The proposed learning system outperforms other models in low-resource languages . balanced datasets and augmenting hateful data with non-hateful examples proved critical .

Copied to clipboard

Leonor Veloso, Hinrich Schuetze

Listening to Affected Communities to Define Extreme Speech: Dataset and Experiments (2022.findings-acl)

Challenge:	Recent studies have focused on mitigating gender bias, but mechanistic interpretations of gender fail to distinguish between factually gendered outputs and gender biased outputs.
Approach:	They propose a benchmark to assess gender knowledge and gender bias in language models . they use neuron ablation to disentangle stereotypical and factual gender .
Outcome:	The proposed benchmark assesses gender knowledge and gender bias in language models across different types of gender-related predictions.

Copied to clipboard

Antonis Maronikolakis, Axel Wisiorek, Leah Nann, Haris Jabbar, Sahana Udupa, Hinrich Schuetze

Multimodality

Challenge:	XTREMESPEECH dataset contains 20,297 social media passages from Brazil, Germany, India and Kenya .
Approach:	They propose a hate speech dataset containing 20,297 social media passages from Brazil, Germany, India and Kenya.
Outcome:	The proposed dataset contains 20,297 social media passages from Brazil, Germany, India and Kenya.

CaMEL: Case Marker Extraction without Labels (2022.acl-long)

Copied to clipboard

Leonie Weissweiler, Valentin Hofmann, Masoud Jalili Sabet, Hinrich Schuetze

Tracing Multilingual Factual Knowledge Acquisition in Pretraining (2025.findings-emnlp)

Challenge:	Existing models for morphological case marking and semantic content are not isomorphic.
Approach:	They propose a model that extracts case markers from a multilingual corpus using a noun phrase chunker and an alignment system.
Outcome:	The proposed model can extract case markers in 83 languages and visualise similarities and differences between case systems and annotate fine-grained deep cases in languages where they are not overtly marked.

Copied to clipboard

Yihong Liu, Mingyang Wang, Amir Hossein Kargaran, Felicia Körner, Ercong Nie, Barbara Plank, François Yvon, Hinrich Schuetze

An Information-Theoretic Approach and Dataset for Probing Gender Stereotypes in Multilingual Masked Language Models (2022.findings-naacl)

Challenge:	Large Language Models are capable of recalling multilingual factual knowledge, but most studies evaluate only the final model, leaving the development of factual recall and crosslingual consistency unexplored.
Approach:	They trace how factual recall and crosslingual consistency evolve during pretraining, focusing on OLMo-7B as a case study.
Outcome:	The results show that fact frequency is the key to a better recall of multilingual facts, regardless of language, and some low-frequency facts in non-English languages can still be correctly recalled.

Copied to clipboard

Victor Steinborn, Philipp Dufter, Haris Jabbar, Hinrich Schuetze

Graph Neural Networks for Multiparallel Word Alignment (2022.findings-acl)

Challenge:	Pretrained language models (PLMs) have been shown to encapsulate social biases, including those relating to gender and race.
Approach:	They propose a new bias measure based on Jensen–Shannon divergence that retains more information from the model output probabilities than other previously proposed bias measures.
Outcome:	The proposed measure outperforms CrowS-Pairs and other similar measures for non-English datasets.

Copied to clipboard

Ayyoob Imani, Lütfi Kerem Senel, Masoud Jalili Sabet, François Yvon, Hinrich Schuetze

Structured Data in NLP

Challenge:	Generally, word alignment algorithms only use bitext and do not make use of the fact that many parallel corpora are multiparallel.
Approach:	They propose a multiparallel word alignment graph and graph neural networks to exploit it . they add and remove edges from the initial alignments and generalize the model .
Outcome:	The proposed method outperforms previous work on three word alignment datasets and on a downstream task.

Look Within or Beyond? A Theoretical Comparison Between Parameter-Efficient and Full Fine-Tuning (2026.acl-long)

Copied to clipboard

YongKang Liu, Xingle Xu, Ercong Nie, Zijing Wang, Shi Feng, Daling Wang, Qian Li, Hinrich Schuetze

PlaM: Training-Free Plateau-Guided Model Merging for Better Visual Grounding in MLLMs (2026.findings-acl)

Challenge:	Parameter-Efficient Fine-Tuning (PEFT) is an alternative to Full-Parameter Fine-tuning, but its effectiveness on complex tasks such as reasoning and instruction-following remains unclear.
Approach:	They propose to use PEFT to reduce the number of trainable parameters while freezing the weights of LLMs.
Outcome:	The proposed methods perform well on standard tasks, but weaknesses on complex and adversarial settings call for new directions beyond current paradigms.

Copied to clipboard

Zijing Wang, YongKang Liu, Mingyang Wang, Ercong Nie, Deyuan Chen, Zhengjie Zhao, Shi Feng, Daling Wang, Xiaocui Yang, Yifei Zhang, Hinrich Schuetze

Multimodality

Challenge:	Multimodal instruction fine-tuning degrades textual reasoning capability, undermining multimodal performance.
Approach:	They propose a plateau-guided model merging method that selectively injects base language model parameters into MLLMs to mitigate this degradation.
Outcome:	The proposed framework reduces multimodal instruction fine-tuning degradation by incorporating a plateau-guided model merging method into MLLMs.

ToPro: Token-Level Prompt Decomposition for Cross-Lingual Sequence Labeling Tasks (2024.eacl-long)

Copied to clipboard

Bolei Ma, Ercong Nie, Shuzhou Yuan, Helmut Schmid, Michael Färber, Frauke Kreuter, Hinrich Schuetze

A Recipe of Parallel Corpora Exploitation for Multilingual Large Language Models (2025.findings-naacl)

Challenge:	Prompt-based methods have been successfully applied to multilingual pretrained language models for zero-shot cross-lingual understanding.
Approach:	They propose a prompt-based method for token-level sequence labeling tasks . they propose to decompose an input sentence into single tokens and apply one prompt template to each token.
Outcome:	The proposed method outperforms Vanilla fine-tuning and Prompt-Tuning in zero-shot cross-lingual transfer . the method also attains state-of-the-art performance when employed with the mT5 model .

Copied to clipboard

Peiqin Lin, Andre Martins, Hinrich Schuetze

Language Mixing in Reasoning Language Models: Patterns, Impact, and Internal Causes (2025.emnlp-main)

Challenge:	Recent studies have highlighted the potential of exploiting parallel corpora to enhance multilingual large language models.
Approach:	They investigate the impact of parallel corpora quality and quantity, training objectives, and model size on performance of multilingual large language models enhanced with parallel corporeal.
Outcome:	The proposed approach improves performance in bilingual and general-purpose tasks.

Copied to clipboard

Mingyang Wang, Lukas Lange, Heike Adel, Yunpu Ma, Jannik Strötgen, Hinrich Schuetze

Differentiable Multi-Agent Actor-Critic for Multi-Step Radiology Report Summarization (2022.acl-long)

Challenge:	Reasoning language models (RLMs) excel at complex tasks by leveraging a chain-of-thought process to generate structured intermediate steps.
Approach:	They present the first systematic study of language mixing in reasoning language models, examining its patterns, impact, and internal causes across 15 languages, 7 task difficulty levels, and 18 subject areas.
Outcome:	The proposed model generates reasoning steps that include a mixture of languages when prompted in one language, and this improves accuracy.

Copied to clipboard

Sanjeev Kumar Karn, Ning Liu, Hinrich Schuetze, Oladimeji Farri

Text Generation

Challenge:	Prior research on radiology report summarization has focused on single-step end-to-end models which subsume the task of salient content acquisition.
Approach:	They propose a two-step extractive summarization followed by abstractive summaries and a new method that breaks down the extractive part into two independent tasks: extraction of salient (1) sentences and (2) keywords.
Outcome:	The proposed model improves on English radiology reports with an overall improvement in F1 score of 3-4% compared to single-step and two-step-with-single-extractive-process baselines.

Crosslingual Transfer Learning for Low-Resource Languages Based on Multilingual Colexification Graphs (2023.findings-emnlp)

Copied to clipboard

Yihong Liu, Haotian Ye, Leonie Weissweiler, Renhao Pei, Hinrich Schuetze

Persistent Personas? Role-Playing, Instruction Following, and Safety in Extended Interactions (2026.eacl-long)

Challenge:	Existing work on colexification patterns relies on annotated word lists, limiting scalability and usefulness in NLP.
Approach:	They propose two methods to train multilingual graphs from colexification patterns using an unannotated parallel corpus.
Outcome:	The proposed methods achieve high recall on CLICS and transfer learning in multilingual graphs.

Copied to clipboard

Pedro Henrique Luz de Araujo, Michael A. Hedderich, Ali Modarressi, Hinrich Schuetze, Benjamin Roth

Dialogue Systems & Conversational Agents

Challenge:	Persona-assigned large language models are used in education, healthcare and sociodemographic simulations.
Approach:	They propose a protocol that combines long persona dialogues and evaluation datasets to create dialogue-conditioned benchmarks that can robustly measure long-context effects.
Outcome:	The proposed protocol can measure persona fidelity, instruction-following, and safety in long conversations.

Breaking the Script Barrier in Multilingual Pre-Trained Language Models with Transliteration-Based Post-Training Alignment (2024.findings-emnlp)

Copied to clipboard

Orgest Xhelili, Yihong Liu, Hinrich Schuetze