Papers by Johannes Bjerva
Text Embedding Inversion Security for Multilingual Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | storing sensitive information as embeddings is susceptible to security breaches, as text can be reconstructed from embeddables . study explores multilingual inversion attacks using a masking defense . |
| Approach: | They propose a simple masking defense that can be used to decode embedded text . they define the problem of black-box multilingual and crosslingual inversion attacks . |
| Outcome: | The proposed defense is effective for both monolingual and multilingual models. |
Quantifying Synthesis and Fusion and their Impact on Machine Translation (2022.naacl-main)
Copied to clipboard
| Challenge: | Literature in Natural Language Processing (NLP) typically labels whole language with strict type of morphology, e.g. fusional or agglutinative. |
| Approach: | They propose to quantify morphological typology at the word and segment level by using two indices: synthesis (e.g. analytic to polysynthetic) and fusion (agglutinative to fusional). |
| Outcome: | The proposed method reduces the rigidity of NLP classification claims by measuring morphological diversity at the word and segment level. |
Uncovering Probabilistic Implications in Typological Knowledge Bases (P19-1)
Copied to clipboard
| Challenge: | linguistic typology is concerned with mapping out the relationships between languages with structural and functional properties. |
| Approach: | They propose a computational model which identifies known and new linguistic universals and uncovers them worthy of further linguistic investigation. |
| Outcome: | The proposed model outperforms baselines and knowledge base baselines. |
From Phonology to Syntax: Unsupervised Linguistic Typology at Different Levels with Language Embeddings (N18-1)
Copied to clipboard
| Challenge: | linguistic typology is the classification of languages according to their linguistic properties. |
| Approach: | They learn distributed language representations which can be used to predict typological properties on a massively multilingual scale. |
| Outcome: | The proposed model can predict typological properties on a massively multilingual scale. |
Transductive Auxiliary Task Self-Training for Neural Multi-Task Models (D19-61)
Copied to clipboard
| Challenge: | Multi-task learning and self-training are two common ways to improve a machine learning model’s performance in settings with limited training data. |
| Approach: | They propose a transductive auxiliary task self-training procedure that trains a model on auxiliary tasks and test instances with auxiliary labels generated by a single-task version of the model. |
| Outcome: | The proposed method improves accuracy by 9.56% over the pure multi-task model for dependency relation tagging and 13.03% for semantic taging. |
What is ”Typological Diversity” in NLP? (2024.emnlp-main)
Copied to clipboard
| Challenge: | linguistic typology is commonly used to motivate language selections, but there are no set definitions or criteria for such claims. |
| Approach: | They propose to use linguistic typology to motivate language selections on the basis that a broad typological sample ought to imply generalization across a wide range of languages. |
| Outcome: | The proposed measures show that skewed language selection can lead to overestimated multilingual performance. |
Zero-Shot Cross-Lingual Transfer with Meta Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | There are more than 7,000 languages spoken in the world, over 90 of which have more than 10 million native speakers each. |
| Approach: | They propose to use meta-learning to train a model on multiple languages at the same time . they use standard supervised, zero-shot cross-lingual, and few-shot crosses-lingual settings for different natural language understanding tasks. |
| Outcome: | The proposed setup improves on the state-of-the-art for a total of 15 languages. |
Large Language Models are Easily Confused: A Quantitative Metric, Security Implications and Typological Analysis (2025.findings-naacl)
Copied to clipboard
| Challenge: | Language Confusion is a phenomenon where Large Language Models (LLMs) generate text that is neither in the desired language, nor in a contextually appropriate one. |
| Approach: | They propose a metric to measure and quantify language confusion in Large Language Models (LLMs) they link language confusion to LLM security and find patterns in the case of multilingual embedding inversion attacks. |
| Outcome: | The proposed metric reveals language confusion across LLMs and link it to LLM security and embedding inversion attacks. |
Does Typological Blinding Impede Cross-Lingual Sharing? (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing work on bridging the performance gap between high- and low-resource languages has only found minor benefits from using typological information. |
| Approach: | They propose to use typological features to train models in a cross-lingual setting to learn latent weights between languages. |
| Outcome: | The proposed model overshadows the utility of explicitly using typological features by ignoring them, and shows that encouraging sharing according to typology improves performance. |
Effective Performance Measurement: Challenges and Opportunities in KPI Extraction from Earnings Calls (2026.acl-industry)
Copied to clipboard
Rasmus T. Aavang, Rasmus Tjalk-Bøggild, Alexandre Iolov, Giovanni Rizzi, Mike Zhang, Johannes Bjerva
| Challenge: | Earnings calls are a key source of financial information about public companies. extracting information from earnings calls is difficult. |
| Approach: | They propose to use LLMs to perform open-ended extraction from unstructured call transcripts to provide a baseline for this valuable domain through the consistent tracking of emergent KPIs. |
| Outcome: | The proposed method provides a baseline for this valuable domain through the consistent tracking of emergent KPIs. |
Linguistically Grounded Analysis of Language Models using Shapley Head Values (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for probing language models for morphosyntactic constructions are not well understood . language models gain knowledge of grammatical phenomena during pretraining, but exactly how this knowledge is encoded is not well established. |
| Approach: | They propose a method for probing language models via Shapley Head Values . they use a BLiMP dataset to test their method on linguistic constructions based on a Shaply Head Value method . |
| Outcome: | The proposed method can be used to investigate linguistic knowledge in language models . it shows that attention heads responsible for processing related linguistic phenomena cluster together . |
Inducing Language-Agnostic Multilingual Representations (2021.starsem-1)
Copied to clipboard
| Challenge: | Cross-lingual representations have the potential to make NLP techniques available to the vast majority of languages in the world, but they currently require large pretraining corpora or access to typologically similar languages. |
| Approach: | They propose to remove language identity signals from multilingual embeddings by re-aligning vector spaces of target languages to a pivot source language and removing language-specific means and variances. |
| Outcome: | The proposed approaches reduce cross-lingual transfer gap by 8.9 points (m-BERT) and 18.2 points (XLM-R) on average across all tasks and languages. |
Parameter sharing between dependency parsers for related languages (D18-1)
Copied to clipboard
| Challenge: | Using parameter sharing between parsers of related languages can improve performance, but there is no consensus on what parameters to share. |
| Approach: | They propose a model where transition classifier parameters are shared and word and character parameters are controlled by a parameter that can be tuned on validation data. |
| Outcome: | The proposed model improves on a monolingually trained baseline. |
ALGEN: Few-shot Inversion Attacks on Textual Embeddings via Cross-Model Alignment and Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Recent studies have proven that private textual data is vulnerable to inversion attacks . authors present a method to reduce the cost of inversion and improve performance . |
| Approach: | They propose a method that aligns victim embeddings to attack space and reconstructs text . they find that none of the defense mechanisms are effective against inversion attacks . |
| Outcome: | The proposed method lowers the cost of inversion and improves performance across languages and domains. |
A Probabilistic Generative Model of Linguistic Typology (N19-1)
Copied to clipboard
| Challenge: | a generative model of languages based on principles-and-parameters posits that languages toggle on or off . linguistic typologists use a set of universal parameters to determine which languages toggle . we show that the correlation between parameters is significant, and that it is not enough to write down the set of parameters available to languages. |
| Approach: | They propose a generative model of language based on exponential-family matrix factorisation. |
| Outcome: | a linguistic model outperforms baseline models on predicting held-out features by exploiting similarities between languages and their features. |
SubjQA: A Dataset for Subjectivity and Review Comprehension (2020.emnlp-main)
Copied to clipboard
| Challenge: | Subjectivity is the expression of internal opinions or beliefs which cannot be objectively observed or verified. |
| Approach: | They develop a dataset which investigates subjectivity in question answering . they find that subjectivity is an important feature in the case of QA . |
| Outcome: | The proposed dataset shows that subjectivity is an important feature in question answering (QA) it also shows that subjective questions and answers can have more complex interactions than previously thought. |
The Past, Present, and Future of Typological Databases in NLP (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Typological information is inconsistent with each other and other sources of typological information, such as linguistic grammars. |
| Approach: | They propose to examine disagreements between typological databases and their uses in NLP by exploring disagreements across databases and resources. |
| Outcome: | The proposed view of typology has significant potential in the future, including in language modeling in low-resource scenarios. |
Shared Path: Unraveling Memorization in Multilingual LLMs through Language Similarities (2025.emnlp-main)
Copied to clipboard
| Challenge: | Using multilingual models, we find that treating languages in isolation obscures the true patterns of memorization. |
| Approach: | They propose a graph-based correlation metric that incorporates language similarity to analyze cross-lingual memorization. |
| Outcome: | The proposed model incorporates language similarity to analyze cross-lingual memorization in 95 languages. |
How Good is Your Wikipedia? Auditing Data Quality for Low-resource and Multilingual NLP (2026.acl-long)
Copied to clipboard
Kushal Tatariya, Artur Kulmizev, Wessel Poelman, Esther Ploeger, Marcel Bollmann, Johannes Bjerva, Jiaming Luo, Heather Lent, Miryam de Lhoneux
| Challenge: | Wikipedia’s perceived high quality and broad language coverage have established it as a fundamental resource in NLP. |
| Approach: | They propose a data filtering procedure which removes a large percentage of Wikipedia's data and a 4-level quality ranking of the site. |
| Outcome: | The results show that the proposed filtering procedure outperforms the raw Wikipedia models in three language modelling scenarios. |
Do LLMs Really Memorize Personally Identifiable Information? Revisiting PII Leakage with a Cue-Controlled Memorization Framework (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been reported to “leak” Personally Identifiable Information (PII) successful PII reconstruction often interpreted as evidence of memorization. |
| Approach: | They propose a principled revision of memorization evaluation for Large Language Models . they propose PII leakage should be evaluated under low lexical cue conditions . |
| Outcome: | The proposed method is based on a multilingual re-evaluation of PII leakage across 32 languages and multiple memorization paradigms. |
Multilingual Gradient Word-Order Typology from Universal Dependencies (2024.eacl-short)
Copied to clipboard
| Challenge: | Existing typological databases, including WALS and Grambank, suffer from inconsistencies due to categorical format. |
| Approach: | They propose a new seed dataset that uses continuous-valued data instead of categorical data to better reflect the variability of language. |
| Outcome: | The proposed dataset can be easily adapted to generate data for a broader set of features and languages. |
Limited-Resource Adapters Are Regularizers, Not Linguists (2025.acl-short)
Copied to clipboard
Marcell Fekete, Nathaniel Romney Robinson, Ernests Lavrinovics, Djeride Jean-Baptiste, Raj Dabre, Johannes Bjerva, Heather Lent
| Challenge: | Existing studies show that cross-lingual transfer from high-resource languages is promising for low-resourced machine translation. |
| Approach: | They propose to use adapter souping and cross-attention fine-tuning to leverage language transfer for Creoles, an under-served group of low-resource languages. |
| Outcome: | The proposed method improves performance over baselines but not meaningfully with adapters. |
Follow the Path: Reasoning over Knowledge Graph Paths to Improve Large Language Model Factuality (2026.findings-acl)
Copied to clipboard
| Challenge: | fs1 improves factuality of reasoning traces by sourcing them from large reasoning models and conditioning them on knowledge graph paths. |
| Approach: | They propose a method that improves the factuality of reasoning traces by sourcing them from large reasoning models and grounding them by conditioning on knowledge graph (KG) paths. |
| Outcome: | The proposed method outperforms instruction-tuned models on open-domain questions . it significantly improves model performance over more complex questions and numerical answer types compared to baselines. |