On the Relation between Linguistic Typology and (Limitations of) Multilingual Language Modeling (D18-1)
Copied to clipboard
| Challenge: | a key challenge in cross-lingual NLP is developing general language-independent architectures that are equally applicable to any language. |
| Approach: | They propose to use a full-vocabulary setup to test the performance of language modeling (LM) on 50 typologically diverse languages. |
| Outcome: | The proposed language modeling task is based on a full vocabulary setup focused on word-level prediction on 50 typologically diverse languages. |
Similar Papers
Morphology Matters: A Multilingual Language Modeling Analysis (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing studies on inflectional morphology disagree on whether or not it makes languages harder to model. |
| Approach: | They propose to use a corpus of 145 Bible translations in 92 languages to investigate whether inflectional morphology makes languages harder to model. |
| Outcome: | The proposed model trains with linguistically motivated subword segmentation strategies and reduces the impact of morphology on language modeling. |
Are All Languages Equally Hard to Language-Model? (N18-2)
Copied to clipboard
| Challenge: | a fair comparison of language models is tricky because of the size of the corpora and the variability of orthographic systems. |
| Approach: | They propose a framework for fair cross-linguistic comparison of language models . they show that in some languages, textual expression is harder to predict with n-gram models compared to LSTM models based on translated text . |
| Outcome: | The proposed framework is based on translated text and language models on 21 languages. |
Assessing the Impact of Typological Features on Multilingual Machine Translation in the Age of Large Language Models (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing evidence on the intrinsic difficulty of multilingual modeling is limited to small monolingual models or bilingual models trained from scratch. |
| Approach: | They propose to use typological properties to determine the difficulty of modeling a language . they analyze two large pre-trained multilingual translation models . |
| Outcome: | The proposed models are based on two large pre-trained models of encoder-decoder and decoder-only machine translation. |
What is ”Typological Diversity” in NLP? (2024.emnlp-main)
Copied to clipboard
| Challenge: | linguistic typology is commonly used to motivate language selections, but there are no set definitions or criteria for such claims. |
| Approach: | They propose to use linguistic typology to motivate language selections on the basis that a broad typological sample ought to imply generalization across a wide range of languages. |
| Outcome: | The proposed measures show that skewed language selection can lead to overestimated multilingual performance. |
Typological Features for Multilingual Delexicalised Dependency Parsing (N19-1)
Copied to clipboard
| Challenge: | Existing universal models to describe the syntax of languages are debated for decades . a new study examines the plausibility of universal grammars in dependency parsing . |
| Approach: | They propose to use typological features to describe the syntax of languages to train a multilingual dependency parser. |
| Outcome: | The proposed model can be trained on 40 languages with the help of typological features. |
From Phonology to Syntax: Unsupervised Linguistic Typology at Different Levels with Language Embeddings (N18-1)
Copied to clipboard
| Challenge: | linguistic typology is the classification of languages according to their linguistic properties. |
| Approach: | They learn distributed language representations which can be used to predict typological properties on a massively multilingual scale. |
| Outcome: | The proposed model can predict typological properties on a massively multilingual scale. |
Show Some Love to Your n-grams: A Bit of Progress and Stronger n-gram Language Modeling Baselines (N19-1)
Copied to clipboard
| Challenge: | Experimental results show that standalone n-gram models lend themselves as natural choices for resource-lean or morphologically rich languages. |
| Approach: | They run experiments on 50 languages covering all morphological language families to compare n-gram models with lstm models. |
| Outcome: | The proposed extension outperforms an lstm language model on 42 languages while its extension which explicitly injects linguistic knowledge outperformed the character-aware neural model on 8 languages. |
Working Hard or Hardly Working: Challenges of Integrating Typology into Neural Dependency Parsers (D19-1)
Copied to clipboard
| Challenge: | linguistic typology has shown great promise in pre-neural parsing, but results for neural architectures have been mixed. |
| Approach: | They explore the task of leveraging typology in the context of cross-lingual dependency parsing. |
| Outcome: | The proposed approach improves performance in the context of cross-lingual dependency parsing. |
What Kind of Language Is Hard to Language-Model? (P19-1)
Copied to clipboard
| Challenge: | a recent study suggests that language models perform poorly across languages. |
| Approach: | They propose a model that fits a paired-sample multiplicative mixed-effects model to obtain language difficulty coefficients from at least-pairwise parallel corpora. |
| Outcome: | The proposed model is able to handle missing data and is aware of inter-sentence variation. |
Modeling Morphological Typology for Unsupervised Learning of Language Morphology (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to morphological analysis relied on hand-built rules to identify word-internal structures. |
| Approach: | They propose a language-independent model for fully unsupervised morphological analysis that exploits a universal framework leveraging morphology. |
| Outcome: | The proposed model outperforms existing systems on nine typologically and genetically diverse languages and shows superior performance over leading systems. |