Inspecting Soundness of AMR Similarity Metrics in terms of Equivalence and Inequivalence (2024.starsem-1)
Copied to clipboard
| Challenge: | Existing Abstract Meaning Representation (AMR) similarity metrics have less investigated their soundness . |
| Approach: | They propose a new experimental method to evaluate soundness of AMR similarity metrics in terms of equivalence and inequivalentity. |
| Outcome: | The proposed method satisfies the soundness criteria of existing AMR similarity metrics and improves them by proposing a revised metric, SMATCH . |
Similar Papers
A Dynamic, Interpreted CheckList for Meaning-oriented NLG Metric Evaluation – through the Lens of Semantic Similarity Rating (2022.starsem-1)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) is a popular NLG evaluation metric . however, it is difficult to judge where exactly such a metric fails . |
| Approach: | They propose a checklist for NLG evaluation metrics that focus on meaning by organizing them around meaning-relevant linguistic phenomena. |
| Outcome: | The proposed metric GraCo computes lexical cohesion graphs over AMR concepts. |
Assessing Polyseme Sense Similarity through Co-predication Acceptability and Contextualised Embedding Distance (2020.starsem-1)
Copied to clipboard
| Challenge: | Co-predication is a commonly used linguistic test to tell apart shifts in polysemic sense from changes in homonymic meaning. |
| Approach: | They examine how co-predication acceptability relates to explicit ratings of polyseme word sense similarity and how well they can be predicted through the distance between target words’ contextualised word embeddings. |
| Outcome: | The proposed measures can be predicted through the distance between target words’ contextualised word embeddings. |
MASSIVE Multilingual Abstract Meaning Representation: A Dataset and Baselines for Hallucination Detection (2024.starsem-1)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) is a semantic formalism that captures the core meaning of an utterance. |
| Approach: | They propose to use AMR to map meanings of 1,685 utterances to 50+ languages to build a dataset 20 times larger than existing resources. |
| Outcome: | The proposed dataset covers more languages, has more utterances, and has localized or translated entities for each language. |
Investigating Aspect Features in Contextualized Embeddings with Semantic Scales and Distributional Similarity (2024.starsem-1)
Copied to clipboard
| Challenge: | Aspect is a linguistic category describing how actions and events unfold over time. |
| Approach: | They propose to use semantic projections to examine whether the vector dimensions of annotated verbs reflect human linguistic distinctions. |
| Outcome: | The proposed models encode the aspects of stativity, durativity and telicity in most of their layers, while durativité is the most challenging feature. |
Disambiguating Emotional Connotations of Words Using Contextualized Word Representations (2024.starsem-1)
Copied to clipboard
| Challenge: | BERT, RoBERTa, XLNet, and GPT-2 models effectively discern emotional connotations of words, demonstrating superior performance and greater resilience against biases. |
| Approach: | They propose to use contextualized word representations to examine how words can be used to distinguish emotional connotations across contexts. |
| Outcome: | The proposed models show that they can distinguish emotional connotations of words in different contexts. |
A Tale of Two Laws of Semantic Change: Predicting Synonym Changes with Distributional Semantic Models (2023.starsem-1)
Copied to clipboard
| Challenge: | Lexical Semantic Change is the study of how the meaning of words evolves through time. |
| Approach: | They propose to use distributional models to detect whether LD or LPC operate for given word pairs. |
| Outcome: | The proposed frameworks achieve a balanced accuracy above 0.6 on the dataset. |
Modeling Sense Structure in Word Usage Graphs with the Weighted Stochastic Block Model (2021.starsem-1)
Copied to clipboard
| Challenge: | Word Usage Graphs capture fine-grained semantic proximity distinctions between word uses. |
| Approach: | They propose to model word use Graphs using a Bayesian weighted stochastic block model and a probabilistic weightes-based model to capture fine-grained semantic proximity distinctions between word uses. |
| Outcome: | The proposed model is compared with existing models and is empirically most adequate. |
Estimating Semantic Similarity between In-Domain and Out-of-Domain Samples (2023.starsem-1)
Copied to clipboard
| Challenge: | Prior work typically defines out-of-domain (OOD) or out- of-distribution (OOdist) samples as those that originate from dataset(s) or source(s), but for the same task. |
| Approach: | They propose to use supervised methods to identify OOD/OODist samples without using a trained model. |
| Outcome: | The proposed method is able to identify OOD/OODist samples without a trained model. |
Topology of Word Embeddings: Singularities Reflect Polysemy (2020.starsem-1)
Copied to clipboard
| Challenge: | a new study suggests that word vectors live on a submanifold within their ambient vector space . a manifold hypothesis suggests that vectors should live on pinched manifels . |
| Approach: | They propose a topological measure of polysemy that correlates well with the actual number of meanings of a word. |
| Outcome: | The proposed method produces competitive results for a word Sense Induction & Disambiguation task. |
Automatic Learning of Modality Exclusivity Norms with Crosslingual Word Embeddings (2020.starsem-1)
Copied to clipboard
| Challenge: | Normative studies on modality for English words are relatively common . however, they are limited to a relatively small number of languages and require costly ratings. |
| Approach: | They aim to learn a mapping between word embeddings and modality norms by training on a high-resource language and testing on . monolingual and crosslingual word embeds are used to predict modality association scores . |
| Outcome: | The proposed model predicts modality associations even when trained on an English resource and tested on a completely unseen language. |