Papers by Jannik Strötgen
Rehearsal-Free Modular and Compositional Continual Learning for Language Models (2024.naacl-short)
Copied to clipboard
| Challenge: | Existing methods to overcome catastrophic forgetting are rehearsal-based and parameter isolation-based. |
| Approach: | They propose a rehearsal-free framework which continuously adds new modules to language models and composes them with existing modules. |
| Outcome: | Experiments on benchmarks show that MoCL outperforms state-of-the-art and effectively facilitates knowledge transfer. |
Better Call SAUL: Fluent and Consistent Language Model Editing with Generation Regularization (2024.findings-emnlp)
Copied to clipboard
| Challenge: | State-of-the-art methods for updating large language models require computational overhead and lack theoretical validation. |
| Approach: | They propose a model editing method that uses sentence concatenation with augmented random facts for generation regularization. |
| Outcome: | The proposed method outperforms state-of-the-art methods while maintaining generation quality and reducing computational overhead. |
Lost in Multilinguality: Dissecting Cross-lingual Factual Inconsistency in Transformer Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Multilingual language models store factual knowledge across languages but struggle to provide consistent responses to semantically equivalent prompts in different languages. |
| Approach: | They propose a linear shortcut method that bypasses computations in the final layers . this method improves accuracy and cross-lingual consistency . |
| Outcome: | The proposed method improves prediction accuracy and cross-lingual consistency. |
Discourse-Aware In-Context Learning for Temporal Expression Normalization (2024.naacl-short)
Copied to clipboard
| Challenge: | Temporal expression (TE) normalization is a well-studied problem, but upcoming machine learning approaches suffer from a lack of labeled data. |
| Approach: | They propose to use in-context learning to inject task, document, and example information into a large language model for temporal expression normalization. |
| Outcome: | The proposed model performs better in non-standard settings by dynamically including relevant examples during inference. |
GradSim: Gradient-Based Language Grouping for Effective Multilingual Training (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that not all languages positively influence each other . multilingual training can help in those cases by sharing knowledge across languages . |
| Approach: | They propose a gradient similarity-based language grouping method for multilingual training that is better correlated with cross-lingual model performance. |
| Outcome: | The proposed method leads to the largest performance gains on a multilingual dataset and is better correlated with cross-lingual model performance. |
To Share or not to Share: Predicting Sets of Sources for Model Transfer Learning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to select transfer sources are limited by text and task similarity, which limits their application in transfer settings where both the task and the text domain change. |
| Approach: | They propose a model similarity measure that represents text and task similarity jointly to automatically determine which and how many sources to exploit. |
| Outcome: | The proposed approach improves performance by 24 F1 points for predicting promising sources across domains and tasks with similar models. |
diaNED: Time-Aware Named Entity Disambiguation for Diachronic Corpora (P18-2)
Copied to clipboard
| Challenge: | Named Entity Disambiguation (NED) systems perform well on news articles but quality drops when inputs span long time periods. |
| Approach: | They propose a time-aware method that resolves ambiguities even when mention contexts give only few cues. |
| Outcome: | The proposed method improves on a newly created diachronic corpus. |
“A Buster Keaton of Linguistics”: First Automated Approaches for the Extraction of Vossian Antonomasia (D19-1)
Copied to clipboard
| Challenge: | Vossian Antonomasia is a stylistic device which attributes a property to a person by naming another person as a reference point. |
| Approach: | They propose a method for the extraction of Vossian Antonomasias that works completely automatically . they use named entity recognition, distant supervision and a bi-directional LSTM . |
| Outcome: | The proposed method outperforms the only existing semi-automatic method for VA identification by more than 30 percentage points in precision. |
A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios (2021.naacl-main)
Copied to clipboard
| Challenge: | a growing body of work is focused on improving performance in low-resource settings . a goal of this study is to explain how these methods differ in their requirements . |
| Approach: | They propose to analyze data-lean scenarios across different dimensions of data availability to understand which approaches are effective in a specific low-resource setting. |
| Outcome: | The proposed methods enable learning when training data is sparse. |
TADA: Efficient Task-Agnostic Domain Adaptation for Transformers (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained transformer-based language models are limited in their expressiveness and domain knowledge. |
| Approach: | They propose a task-agnostic domain adaptation method which is modular, parameter-efficient, and data-efficient. |
| Outcome: | The proposed method is efficient and modular, parameter-efficient, and data-efficient. |
FAME: Feature-Based Adversarial Meta-Embeddings for Robust Input Representations (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on word embeddings and pre-trained language models has shown the large impact of language representations on natural language processing (NLP) models across tasks and domains. |
| Approach: | They propose feature-based adversarial meta-embeddings with an attention function that is guided by word-specific properties, such as shape and frequency, to handle subword-based embeddings. |
| Outcome: | The proposed model improves performance in downstream tasks even with word embeddings from transformers. |
NLNDE: Enhancing Neural Sequence Taggers with Attention and Noisy Channel for Robust Pharmacological Entity Detection (D19-57)
Copied to clipboard
| Challenge: | Named entity recognition has been extensively studied on English news texts, but transfer to other domains and languages is still a challenging problem. |
| Approach: | They propose a system that provides a non-standard domain and language setting for pharmacological entity detection in Spanish texts and a sequencelabeling task that requires neither language nor domain expertise. |
| Outcome: | The proposed system achieves up to 88.6% F1 in the PharmaCoNER competition and is based on a sequence labeling task and training on annotated data. |
Language Mixing in Reasoning Language Models: Patterns, Impact, and Internal Causes (2025.emnlp-main)
Copied to clipboard
| Challenge: | Reasoning language models (RLMs) excel at complex tasks by leveraging a chain-of-thought process to generate structured intermediate steps. |
| Approach: | They present the first systematic study of language mixing in reasoning language models, examining its patterns, impact, and internal causes across 15 languages, 7 task difficulty levels, and 18 subject areas. |
| Outcome: | The proposed model generates reasoning steps that include a mixture of languages when prompted in one language, and this improves accuracy. |
Closing the Gap: Joint De-Identification and Concept Extraction in the Clinical Domain (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies show that de-identification is effective in the clinical domain but not in the downstream tasks. |
| Approach: | They propose a stacked model with restricted access to privacy sensitive information and a multitask model to investigate the effect of de-identification on clinical concept extraction. |
| Outcome: | The proposed model is stacked with restricted access to privacy sensitive information and a multitask model. |
Three Real-World Datasets and Neural Computational Models for Classification Tasks in Patent Landscaping (2022.emnlp-main)
Copied to clipboard
| Challenge: | Patent Landscaping is one of the central tasks of intellectual property management and involves selecting and grouping patents according to user-defined technical or application-oriented criteria. |
| Approach: | They propose to use a novel model that takes into account textual information from the patents’ full texts as well as embeddings created based on the patent’s CPC labels. |
| Outcome: | The proposed model takes into account textual information from the patents’ full texts as well as embeddings created based on the patent’s CPC labels. |
KRAUTS: A German Temporally Annotated News Corpus (L18-1)
Copied to clipboard
| Challenge: | Temporal tagging is an important task towards improved natural language understanding. |
| Approach: | They present a new German temporally annotated corpus with 192 documents with 1,140 annotations . they propose to make temporal tagging a viable research area . |
| Outcome: | The proposed corpus contains 192 documents with 1,140 annotated temporal expressions. |
Multilingual Normalization of Temporal Expressions with Masked Language Models (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for normalizing temporal expressions are rule-based, which severely limits the applicability in multilingual settings. |
| Approach: | They propose a neural method for normalizing temporal expressions based on masked language modeling and a slot-based prediction scheme for context-independent representations. |
| Outcome: | The proposed method outperforms existing rule-based methods in many languages and in particular, for low-resource languages with performance improvements of up to 33 F1 on average compared to the state of the art. |