A Crosslingual Investigation of Conceptualization in 1335 Languages (2023.acl-long)
Copied to clipboard
Yihong Liu, Haotian Ye, Leonie Weissweiler, Philipp Wicke, Renhao Pei, Robert Zangenfeind, Hinrich Schütze
| Challenge: | Conceptualizer is a method that creates a bipartite directed alignment graph between source language concepts and sets of target language strings. |
| Approach: | They propose a method that creates a bipartite directed alignment graph between source language concepts and sets of target language strings. |
| Outcome: | The proposed method has good alignment accuracy across all languages and on 32 Swadesh concepts. |
Similar Papers
Understanding Cross-Lingual Alignment—A Survey (2024.findings-acl)
Copied to clipboard
| Challenge: | Cross-lingual alignment is the meaningful similarity of representations across languages in multilingual language models. |
| Approach: | They propose a taxonomy of methods to improve cross-lingual alignment . they argue that an effective trade-off between language-neutral and language-specific information is key . |
| Outcome: | The proposed methods can be applied to encoder models and encoder-decoder-only models . they show that language-neutral and language-specific information is key . |
Exploring Alignment in Shared Cross-lingual Spaces (2024.acl-long)
Copied to clipboard
| Challenge: | a new study examines the degree of alignment between languages in multilingual embeddings . cross-lingual embeds are designed to encode linguistic concepts that bridge equivalent semantic meaning . a comprehensive approach is needed to address these questions. |
| Approach: | They employ clustering to uncover latent concepts within multilingual models . they introduce two metrics to quantify alignment and overlap of these concepts . |
| Outcome: | The proposed model can capture linguistic nuances across languages, but is not language-agnostic? the proposed model is able to capture nuances in multiple languages, the authors say. |
Are Structural Concepts Universal in Transformer Language Models? Towards Interpretable Cross-Lingual Generalization (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have implicitly transfer knowledge across languages, but not all languages have such generalization capabilities. |
| Approach: | They propose a meta-learning-based method to learn to align conceptual spaces of different languages to enhance cross-lingual generalization. |
| Outcome: | The proposed method achieves competitive results with state-of-the-art methods and narrows the performance gap between languages. |
Locally Measuring Cross-lingual Lexical Alignment: A Domain and Word Level Perspective (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a cognitive science research focus on aligning language spaces in their entirety . but, cognitive science has long focused on a local perspective . a new method for cross-lingual lexical alignment requires some methodology . |
| Approach: | They propose a method for analyzing kinship domain kinematics and a new method for contextualization . they propose kin-level validations and contextualizations to validate the results . |
| Outcome: | The proposed method analyzes synthetic validations and naturalistic validations using lexical gaps in the kinship domain. |
A Massively Multilingual Analysis of Cross-linguality in Shared Embedding Space (2021.emnlp-main)
Copied to clipboard
| Challenge: | Cross-lingual language models house representations for many different languages in the same space. |
| Approach: | They investigate linguistic and non-linguistic factors affecting sentence-level alignment in cross-lingual pretrained language models for 101 languages and 5,050 language pairs. |
| Outcome: | The results show that word order agreement and agreement in morphological complexity are strongest predictors of cross-linguality. |
Exploring Multilingual Concepts of Human Values in Large Language Models: Is Value Alignment Consistent, Transferable and Controllable across Languages? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Prior research has revealed that certain abstract concepts are linearly represented as directions in the representation space of LLMs, predominantly centered around English. |
| Approach: | They extend previous research that shows certain abstract concepts are linearly represented as directions in LLMs, predominantly centered around English. |
| Outcome: | The proposed model can be used to align LLMs with human values, and it can generate toxic, untruthful, biased, and even illegal content. |
Concept Space Alignment in Multilingual LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual large language models generalize somewhat across languages, but it is unclear whether this is a result of improved, implicit alignment, or of something else, e.g., linguistic overlap or semi-parallel subsets of training data. |
| Approach: | They hypothesize that implicit alignment is the reason for generalization in multilingual large language models. |
| Outcome: | The proposed model generalizes well across languages, but lacks linearity. |
A Corpus for Multilingual Document Classification in Eight Languages (L18-1)
Copied to clipboard
| Challenge: | a subset of the Reuters corpus volume 2 is used to evaluate cross-lingual document classification . current best practice is to evaluate document classification on resources in one language and transfer it to another without additional resources. |
| Approach: | They propose to use a subset of the Reuters corpus to evaluate cross-lingual document classification . they propose to add Italian, Russian, Japanese and Chinese to the subset . |
| Outcome: | The proposed subset of the Reuters corpus has balanced class priors for eight languages. |
A Call for More Rigor in Unsupervised Cross-lingual Learning (2020.acl-main)
Copied to clipboard
| Challenge: | Existing research on unsupervised cross-lingual learning has focused on purely unsupervised learning without any parallel data for most of the world's languages. |
| Approach: | They propose to define "multilingual learning" as learning a common model for two or more languages from raw text, without any downstream task labels. |
| Outcome: | The proposed model is based on a model with no parallel data and abundant monolingual data. |
Finding Concept-specific Biases in Form–Meaning Associations (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to detect cross-linguistic associations are not effective, but their effects are minor. |
| Approach: | They propose a method to measure cross-linguistic associations by controlling for the influence of language family and geographic proximity within a large concept-aligned, cross-lingual lexicon. |
| Outcome: | The proposed method shows that it is small, but it is unsurprisingly small (less than 0.5% on average). |