FOOL ME IF YOU CAN! An Adversarial Dataset to Investigate the Robustness of LMs in Word Sense Disambiguation (2024.emnlp-main)
Copied to clipboard
Mohamad Ballout, Anne Dedert, Nohayr Abdelmoneim, Ulf Krumnack, Gunther Heidemann, Kai-Uwe Kühnberger
| Challenge: | Word sense disambiguation (WSD) is a key task in natural language processing . however, these models struggle with recognizing semantic boundaries in adversarial contexts . |
| Approach: | They propose to use a coarse-grained WSD dataset to assess model robustness . they found that some models struggled to correctly disambiguate homonyms in adversarial contexts . |
| Outcome: | The proposed dataset includes four test sets to assess the robustness of language models in WSD tasks. |
Similar Papers
Nibbling at the Hard Core of Word Sense Disambiguation (2022.acl-long)
Copied to clipboard
| Challenge: | Word Sense Disambiguation (WSD) is a task that is based on a set of pre-trained language models. |
| Approach: | They propose to use Word Sense Disambiguation to test whether systems can handle ambiguous words. |
| Outcome: | The proposed benchmarks show that seven of the most representative state-of-the-art systems make trivial errors on traditional evaluation benchmarks. |
RoDEval: A Robust Word Sense Disambiguation Evaluation Framework for Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies rely on single-task evaluations and classification-based metrics that overlook the fundamental differences between generative LLMs and traditional classification models. |
| Approach: | They propose to use four new metrics to evaluate LLM-based word sense disambiguation (WSD) . experimental results reveal significant limitations in LLMs' WSD performance . |
| Outcome: | The proposed evaluation framework is open-source at https://github.com/DayDream405/RoDEval. |
Analyzing Homonymy Disambiguation Capabilities of Pretrained Language Models (2024.lrec-main)
Copied to clipboard
Lorenzo Proietti, Stefano Perrella, Simone Tedeschi, Giulia Vulpis, Leonardo Lavalle, Andrea Sanchietti, Andrea Ferrari, Roberto Navigli
| Challenge: | Word Sense Disambiguation (WSD) is a key task in Natural Language Processing (NLP) but current pretrained language models lack the granularity to perform disambiguation . |
| Approach: | They propose a large-scale resource that leverages homonymy relations to cluster WordNet senses and train Homonymy Disambiguation systems. |
| Outcome: | The proposed model can distinguish homonyms with up to 95% accuracy even without fine-tuning the underlying PLM. |
How Much Do Encoder Models Know About Word Senses? (2025.acl-long)
Copied to clipboard
| Challenge: | Word Sense Disambiguation (WSD) is a key task in Natural Language Processing (NLP) however, how well these models inherently disambiguate word senses remains uncertain. |
| Approach: | They evaluate several encoder-only PLMs across WordNet and ODE sense inventories to evaluate their ability to separate word senses without any task-specific fine-tuning. |
| Outcome: | The proposed model outperforms output layer on WordNet and ODE sense inventories by 15 percentage points. |
MADAWSD: Multi-Agent Debate Framework for Adversarial Word Sense Disambiguation (2025.emnlp-main)
Copied to clipboard
Kaiyuan Zhang, Qian Liu, Luyang Zhang, Chaoqun Zheng, Shuaimin Li, Bing Xu, Muyun Yang, Xinxiao Qiao, Wenpeng Lu
| Challenge: | Word sense disambiguation (WSD) is a fundamental yet challenging task in natural language processing. |
| Approach: | a novel multi-agent Debate framework for adversarial word Sense disambiguation is proposed . the framework simulates a real-world debate environment where multiple agents engage in discussions about ambiguous words in the context of adversarials. |
| Outcome: | The proposed framework integrates with existing LLMs and improves models in Chinese language . it shows that it can be used to improve models in the Chinese language and improve performance . |
Improved Word Sense Disambiguation Using Pre-Trained Contextualized Word Representations (D19-1)
Copied to clipboard
| Challenge: | Contextualized word representations are effective in downstream tasks such as question answering, named entity recognition, and sentiment analysis. |
| Approach: | They propose to integrate pre-trained contextualized word representations into a neural network that captures the whole sentence and the word representation in the sentence. |
| Outcome: | The proposed approach outperforms the state-of-the-art approach that makes use of non-contextualized word embeddings on multiple benchmark WSD datasets. |
Towards General-Domain Word Sense Disambiguation: Distilling Large Language Model into Compact Disambiguator (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for Word Sense Disambiguation rely heavily on manually annotated data, which limits coverage and generalization. |
| Approach: | They propose a framework that leverages large language models as knowledge distillers to build silver-standard WSD corpora by combining generation-based distillation and annotation-based disambiguation. |
| Outcome: | The proposed framework outperforms existing methods on general-domain benchmarks by 50% on the most challenging test set and by 1000 times fewer parameters. |
SANDWiCH: Semantical Analysis of Neighbours for Disambiguating Words in Context ad Hoc (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent studies show that language understanding offered by chat-based Large Language Models is limited and far from human-like performance. |
| Approach: | They propose a framework for multilingual Word Sense Disambiguation using group algebra. |
| Outcome: | The proposed framework surpasses the performance of current alternatives even in low-resource languages while reducing the parameter count by 72%. |
Enhancing the Context Representation in Similarity-based Word Sense Disambiguation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing similarity-based systems focus on learning sense embeddings using only the sentence where the word appears, neglecting its global context. |
| Approach: | They propose a contextoriented embedding technique that takes better advantage of both word-level and sense-level global context of an ambiguous word for disambiguation. |
| Outcome: | The proposed method improves on all-words WSD benchmarks in knowledge-based category by large margins. |
A Deep Dive into Word Sense Disambiguation with LSTM (C18-1)
Copied to clipboard
| Challenge: | LSTM-based language models have been shown effective in Word Sense Disambiguation (WSD) but neither the training data nor the source code was released. |
| Approach: | They propose to use LSTM-based language models to perform Word Sense Disambiguation (WSD) using openly available datasets and software. |
| Outcome: | The proposed method returned state-of-the-art performance in several benchmarks, but neither the training data nor the source code were released. |