| Challenge: | Existing studies on inflectional morphology disagree on whether or not it makes languages harder to model. |
| Approach: | They propose to use a corpus of 145 Bible translations in 92 languages to investigate whether inflectional morphology makes languages harder to model. |
| Outcome: | The proposed model trains with linguistically motivated subword segmentation strategies and reduces the impact of morphology on language modeling. |
Similar Papers
What Kind of Language Is Hard to Language-Model? (P19-1)
Copied to clipboard
| Challenge: | a recent study suggests that language models perform poorly across languages. |
| Approach: | They propose a model that fits a paired-sample multiplicative mixed-effects model to obtain language difficulty coefficients from at least-pairwise parallel corpora. |
| Outcome: | The proposed model is able to handle missing data and is aware of inter-sentence variation. |
On the Relation between Linguistic Typology and (Limitations of) Multilingual Language Modeling (D18-1)
Copied to clipboard
| Challenge: | a key challenge in cross-lingual NLP is developing general language-independent architectures that are equally applicable to any language. |
| Approach: | They propose to use a full-vocabulary setup to test the performance of language modeling (LM) on 50 typologically diverse languages. |
| Outcome: | The proposed language modeling task is based on a full vocabulary setup focused on word-level prediction on 50 typologically diverse languages. |
Why do language models perform worse for morphologically complex languages? (2025.coling-main)
Copied to clipboard
| Challenge: | Language models perform differently across languages, a new study suggests . morphological typology may explain some of the performance differences, authors say . |
| Approach: | They propose to test morphological alignment of tokenizers, tokenization quality and disparities in dataset sizes and measurement to test this hypothesis. |
| Outcome: | The proposed model shows that fusional languages perform better than fusionative languages . the authors suggest that morphological typology may explain some of the performance differences . |
Confounding Factors in Relating Model Performance to Morphology (2025.emnlp-main)
Copied to clipboard
| Challenge: | morphological differences between languages are unclear, but are often considered unimportant . confounding factors make it hard to compare results and draw conclusions, authors argue . |
| Approach: | They propose to use token bigram metrics to predict difficulty of causal language modeling . they argue that confounding factors are contributing to the conflicting evidence . |
| Outcome: | The proposed metrics better capture the relation between morphology and tokenization compared to word-based models. |
Are All Languages Equally Hard to Language-Model? (N18-2)
Copied to clipboard
| Challenge: | a fair comparison of language models is tricky because of the size of the corpora and the variability of orthographic systems. |
| Approach: | They propose a framework for fair cross-linguistic comparison of language models . they show that in some languages, textual expression is harder to predict with n-gram models compared to LSTM models based on translated text . |
| Outcome: | The proposed framework is based on translated text and language models on 21 languages. |
Unlike “Likely”, “Unlike” is Unlikely: BPE-based Segmentation hurts Morphological Derivations in LLMs (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) use subword vocabularies to process and generate text. |
| Approach: | They find that Large Language Models (LLMs) perform poorly at handling some types of affixations because subwords are marked as initial- or intra-word . |
| Outcome: | The largest models trained on enough data can mitigate this tendency because initial- and intra-word embeddings are aligned; in-context learning also helps when all examples are selected in a consistent way; but only morphological segmentation can achieve a near-perfect accuracy. |
Exploring Linguistic Probes for Morphological Inflection (2023.emnlp-main)
Copied to clipboard
| Challenge: | morphological inflection models typically employ language-independent data splitting algorithms. |
| Approach: | They propose language-specific probes to test aspects of morphological generalization . they use three morphology-distinct languages to test their generalization abilities . |
| Outcome: | The proposed language-specific probes are used to test morphological generalization abilities on three distinct languages. |
A Morphology-Based Investigation of Positional Encodings (2024.emnlp-main)
Copied to clipboard
| Challenge: | Contemporary deep learning models handle languages with diverse morphology . morphological complexity of languages is closely linked with positional encodings . |
| Approach: | They propose to use positional encodings to integrate morphological complexity into deep learning models. |
| Outcome: | The proposed model improves on 22 languages and 5 downstream tasks. |
Morphology Without Borders: Clause-Level Morphology (2022.tacl-1)
Copied to clipboard
| Challenge: | Morphological tasks use large multi-lingual datasets that organize words into inflection tables . lack of a clear linguistic and operational definition of what is a word impairs universality of tasks . |
| Approach: | They propose to view morphology as a clause-level phenomenon, rather than word-level . they propose to use a dataset for clause- level morphological tasks in 4 different languages . |
| Outcome: | The proposed dataset for clause-level morphology covers 4 typologically different languages: English, German, Turkish, and Hebrew. |
BPE vs. Morphological Segmentation: A Case Study on Machine Translation of Four Polysynthetic Languages (2022.findings-acl)
Copied to clipboard
| Challenge: | Morphologically rich polysynthetic languages present a challenge for NLP systems due to data sparsity. |
| Approach: | They propose to use subword segmentation to reduce data sparsity in polysynthetic languages . they compare supervised and unsupervised morphological segmentation methods to Byte-Pair Encodings . |
| Outcome: | The proposed methods outperform BPEs in MT tasks for all language pairs except for Nahuatl . the proposed methods are more efficient than supervised methods, but less sparse in fusional languages. |