AM2iCo: Evaluating Word Meaning in Context across Low-Resource Languages with Adversarial Examples (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing multilingual evaluation datasets that evaluate lexical semantics "in-context" have various limitations, including limited coverage of high-resource languages and superficial cues. |
| Approach: | They propose to use a set of pretrained language models to evaluate lexical semantics in context. |
| Outcome: | The proposed set shows that current models lag behind human performance in interpreting word meaning in cross-lingual contexts. |
Similar Papers
Exploring the Representation of Word Meanings in Context: A Case Study on Homonymy and Synonymy (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models that represent different senses of words in context are not accurate for polysemous words. |
| Approach: | They propose a multilingual dataset that evaluates the ability of models to accurately represent different lexical-semantic relations such as homonymy and synonymy. |
| Outcome: | The proposed models can disambiguate homonyms in context, but fail to represent words with different senses when occurring in similar sentences. |
Bootstrapping Multilingual AMR with Contextual Word Alignments (2021.eacl-main)
Copied to clipboard
Janaki Sheth, Young-Suk Lee, Ramón Fernandez Astudillo, Tahira Naseem, Radu Florian, Salim Roukos, Todd Ward
| Challenge: | Abstract Meaning Representation (AMR) is a sentence-level graph that is biased towards English. |
| Approach: | They propose a technique for foreign-text-to-English AMR alignment using contextual word alignment between English and foreign language tokens. |
| Outcome: | The proposed technique outperforms the best results for German, Italian, Spanish and Chinese. |
Combating the Curse of Multilinguality in Cross-Lingual WSD by Aligning Sparse Contextualized Word Representations (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to handle knowledge acquisition bottlenecks in multilingual training are limited due to the curse of multilinguality. |
| Approach: | They propose to use large pre-trained monolingual language models in cross lingual zero-shot word sense disambiguation coupled with a contextualized mapping mechanism. |
| Outcome: | The proposed model improves the average F-score by nearly 6.5 points over 17 target languages. |
DWUG: A large Resource of Diachronic Word Usage Graphs in Four Languages (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for graded contextual word meaning annotation have not been implemented yet. |
| Approach: | They propose a multi-round incremental annotation process and a clustering algorithm to group usages into senses to create a large-scale dataset. |
| Outcome: | The proposed method is the largest resource of graded contextualized, diachronic word meaning annotation in four different languages, based on 100,000 human semantic proximity judgments. |
Cross-lingual Multi-Level Adversarial Transfer to Enhance Low-Resource Name Tagging (N19-1)
Copied to clipboard
| Challenge: | Low-resource language name tagging is an important but challenging task. |
| Approach: | They propose a neural architecture that leverages multi-level adversarial transfer to improve name tagging for low-resource languages. |
| Outcome: | The proposed approach outperforms previous approaches on CoNLL data sets. |
Accounting for Language Effect in the Evaluation of Cross-lingual AMR Parsers (2022.coling-1)
Copied to clipboard
| Challenge: | Existing multilingual AMR evaluation metrics are not available for cross-lingual parsers . existing studies show that source language has a dramatic effect on cross-linguistic AMRs . |
| Approach: | They propose to use three multilingual adaptations of monolingual AMR evaluation metrics to evaluate cross-lingual AML parsers. |
| Outcome: | The proposed metric is the most highly correlated to english AMRs, while the most correlated is S2match. |
Evaluating Contextualized Representations of (Spanish) Ambiguous Words: A New Lexical Resource and Empirical Analysis (2025.naacl-long)
Copied to clipboard
| Challenge: | Few studies have systematically compared LMs’ contextualized word embeddings for languages beyond English. |
| Approach: | They evaluate Spanish ambiguous nouns in context in a suite of Spanish-language monolingual and multilingual BERT-based models. |
| Outcome: | The proposed model captures some variance in human relatedness judgments but falls short of the human benchmark. |
Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation (2025.acl-long)
Copied to clipboard
Shivalika Singh, Angelika Romanou, Clémentine Fourrier, David Ifeoluwa Adelani, Jian Gang Ngui, Daniel Vila-Suero, Peerat Limkonchotiwat, Kelly Marchisio, Wei Qi Leong, Yosephine Susanto, Raymond Ng, Shayne Longpre, Sebastian Ruder, Wei-Yin Ko, Antoine Bosselut, Alice Oh, Andre Martins, Leshem Choshen, Daphne Ippolito, Enzo Ferrante, Marzieh Fadaee, Beyza Ermis, Sara Hooker
| Challenge: | Reliable multilingual evaluation is difficult and culturally appropriate evaluation is even harder to achieve. |
| Approach: | They propose a multilingual evaluation framework that aims to mitigate these biases by improving translations and annotation practices. |
| Outcome: | The proposed framework improves translation quality and cultural coverage and is culturally sensitive and culturally agnostic. |
Speakers Fill Lexical Semantic Gaps with Context (2020.emnlp-main)
Copied to clipboard
| Challenge: | Lexical ambiguity is widespread in language, allowing for the reuse of economical word forms and thus making language more efficient. |
| Approach: | They propose two ways to estimate lexical ambiguity as the entropy of meanings a word can take . they validate this hypothesis by using WordNet and BERT . |
| Outcome: | The proposed method shows that on six high-resource languages, there are significant correlations between the estimate and the number of synonyms a word has in WordNet. |
Do Context-Aware Translation Models Pay the Right Attention? (2021.acl-long)
Copied to clipboard
| Challenge: | Context-aware machine translation models fail to leverage contextual information to resolve ambiguous words and pronouns. |
| Approach: | They propose a new dataset that includes supporting context words for 14K translations that professional translators found useful for pronoun disambiguation. |
| Outcome: | The proposed model can automatically disambiguate pronouns and polysemous words when they are not in the same context. |