| Challenge: | In this paper, we fine-tune pretrained deep nets such as BERT and ERNIE . at inference time, these nets can be used to distinguish synonyms from antonyms . |
| Approach: | They propose to use lexical resources to fine-tune pretrained deep nets such as BERT and ERNIE to distinguish synonyms from antonyms. |
| Outcome: | The proposed method can be applied to multiword expressions, out of vocabulary words, morphological variants and more. |
Similar Papers
Inferences for Lexical Semantic Resource Building with Less Supervision (2020.lrec-1)
Copied to clipboard
| Challenge: | lexical semantic resources may be built using various approaches such as extraction from corpora, integration of relevant pieces of knowledge from pre-existing knowledge resources and endogenous inference. |
| Approach: | They propose a method where the resource building process appears as a self learning process . they propose lexical and semantic resource building based on inference . |
| Outcome: | The proposed method reduces the human effort needed for lexical semantic resource building. |
Probing Pretrained Language Models for Lexical Semantics (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have focused on morphosyntactic, semantic, and world knowledge, but it remains unclear to what extent LMs derive lexical type-level knowledge from words in context. |
| Approach: | They propose to use multilingual and monolingual LMs to extract lexical type-level knowledge from words in context. |
| Outcome: | The proposed models perform well across six typologically diverse languages and five lexical tasks. |
BERT-based Lexical Substitution (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to lexical substitution tend to overlook good substitute candidates that are not the synonyms of the target words in the lexicals and fail to take into account the substitution’s influence on the global context of the sentence. |
| Approach: | They propose an end-to-end BERT-based lexical substitution approach which proposes and validates substitute candidates without using annotated data or manually curated resources. |
| Outcome: | The proposed approach performs well in proposing and ranking substitute candidates, achieving the state-of-the-art results in both LS07 and LS14 benchmarks. |
A Survey on Automatically-Constructed WordNets and their Evaluation: Lexical and Word Embedding-based Approaches (L18-1)
Copied to clipboard
| Challenge: | WordNets are lexical databases in which groups of synonyms are stored according to the semantic relationships between them. |
| Approach: | This paper describes various approaches to constructing WordNets automatically by leveraging traditional lexical resources and newer trends such as word embeddings. |
| Outcome: | The proposed methods leverage traditional lexical resources and newer trends such as word embeddings to build and evaluate WordNets. |
Evaluating Pretrained Causal Language Models for Synonymy (2025.findings-acl)
Copied to clipboard
| Challenge: | Despite the scaling of causal language models, the underlying basis of complex skills remains unclear. |
| Approach: | They propose that subjacent skills such as synonymy might be explained using linguistic concepts. |
| Outcome: | The proposed model recognizes synonymy but struggles to generate synonyms when prompted with relevant context. |
Can You Tell Me How to Get Past Sesame Street? Sentence-Level Pretraining Beyond Language Modeling (P19-1)
Copied to clipboard
Alex Wang, Jan Hula, Patrick Xia, Raghavendra Pappagari, R. Thomas McCoy, Roma Patel, Najoung Kim, Ian Tenney, Yinghui Huang, Katherin Yu, Shuning Jin, Berlin Chen, Benjamin Van Durme, Edouard Grave, Ellie Pavlick, Samuel R. Bowman
| Challenge: | State-of-the-art models in natural language processing (NLP) often incorporate sentence encoder functions which generate a sequence of vectors intended to represent the in-context meaning of each word in an input text. |
| Approach: | They conduct the first large-scale systematic study of candidate pretraining tasks, comparing 19 different tasks as alternatives and complements to language modeling. |
| Outcome: | The proposed model can be used to train sentences on language modeling tasks. |
LexFit: Lexical Fine-Tuning of Pretrained Language Models (2021.acl-long)
Copied to clipboard
| Challenge: | Transformer-based language models implicitly store a wealth of lexical semantic knowledge, but it is non-trivial to extract that knowledge effectively from their parameters. |
| Approach: | They propose to expose and enrich lexical knowledge from transformer-based language models to serve as effective decontextualized word encoders even when fed input words "in isolation" |
| Outcome: | The proposed model outperforms standard static WEs and vanilla LMs in lexical tasks over four established tasks in 8 languages. |
Scientific and Creative Analogies in Pretrained Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing analogy datasets focus on a limited set of analogical relations with a high similarity of the two domains between which the analogy holds. |
| Approach: | They propose a dataset that encodes analogy in pretrained language models . they use a system that maps attributes and relational structures across dissimilar domains . |
| Outcome: | The proposed dataset shows that state-of-the-art models achieve low performance on analogy tasks . |
Enhancing Word Embeddings with Knowledge Extracted from Lexical Resources (2020.acl-srw)
Copied to clipboard
| Challenge: | In this paper, we present an effective method for semantic specialization of word vector representations. |
| Approach: | They propose a method for semantic specialization of word vector representations using BabelNet. |
| Outcome: | The proposed method improves on word similarity and dialog state tracking tasks. |
Code Synonyms Do Matter: Multiple Synonyms Matching Network for Automatic ICD Coding (2022.acl-short)
Copied to clipboard
| Challenge: | Existing methods for automatic ICD coding use label attention to match related text snippets. |
| Approach: | They propose to use code synonyms to leverage for better code representation learning. |
| Outcome: | The proposed method outperforms previous state-of-the-art methods on the MIMIC-III dataset. |