Assessing the Limits of the Distributional Hypothesis in Semantic Spaces: Trait-based Relational Knowledge and the Impact of Co-occurrences (2022.starsem-1)
Copied to clipboard
| Challenge: | a rise in performance in NLP has led to a decrease in interpretability . a recent study examined how neural semantic models capture relational knowledge . |
| Approach: | They evaluate how well English and Spanish semantic spaces capture a particular type of relational knowledge . they also explore the role of co-occurrences in this context . |
| Outcome: | The proposed model can be used to predict traits associated with concepts in English and Spanish. |
Similar Papers
How Does Data Corruption Affect Natural Language Understanding Models? A Study on GLUE datasets (2022.starsem-1)
Copied to clipboard
| Challenge: | Existing studies on the performance of pre-trained language models on natural language understanding tasks have focused on the natural language inference and textual entailment tasks. |
| Approach: | They propose to use corrupted data to fine-tune pre-trained language models to assess their language understanding capabilities. |
| Outcome: | The proposed transformations can be applied to all but one NLU task and show that understanding the meaning of utterances is not required for high performance. |
Did the Cat Drink the Coffee? Challenging Transformers with Generalized Event Knowledge (2021.starsem-1)
Copied to clipboard
Paolo Pedinotti, Giulia Rambelli, Emmanuele Chersoni, Enrico Santus, Alessandro Lenci, Philippe Blache
| Challenge: | Prior work has explored the ability of computational models to predict word semantic fit with a given predicate. |
| Approach: | They compare Transformers Language Models to SDM to assess their performance . they found that TLMs do not capture important aspects of event knowledge . people can discriminate between typical and atypical events, they say . |
| Outcome: | The proposed models can achieve comparable performance to SDM, but they lack important aspects of event knowledge. |
Limits for learning with language models (2023.starsem-1)
Copied to clipboard
| Challenge: | Recent studies show that large language models fail to capture important aspects of linguistic meaning . authors argue that LLMs cannot learn fundamental semantic properties defined in formal semantics . |
| Approach: | They propose a theoretical explanation for some of the observed failings of large language models . they show that LLMs cannot learn certain fundamental semantic properties . |
| Outcome: | The proposed model fails to learn semantic entailment and consistency as defined in formal semantics, the authors argue . their model fails on tasks that require engorgements and deep linguistic understanding, they argue - but not on universal quantification. |
Measuring Alignment Bias in Neural Seq2seq Semantic Parsers (2022.starsem-1)
Copied to clipboard
| Challenge: | Sequence-to-sequence semantic parsers with attention mechanisms have changed the research landscape . emergence of seq2seq models have led to questions about alignments . |
| Approach: | They investigate whether seq2seq models can handle both simple and complex alignments. |
| Outcome: | The proposed model performs better on monotonic and complex alignments compared to monotonic models . |
Overcoming Poor Word Embeddings with Word Definitions (2021.starsem-1)
Copied to clipboard
| Challenge: | Modern natural language understanding models depend on pretrained word embeddings, but applications may need to reason about words that were never or rarely seen during pretraining. |
| Approach: | They propose a method to improve a model's ability to learn to use definitions in natural text to overcome this handicap. |
| Outcome: | The proposed model learns to use definitions in natural text to overcome this handicap. |
One Semantic Parser to Parse Them All: Sequence to Sequence Multi-Task Learning on Semantic Parsing Datasets (2021.starsem-1)
Copied to clipboard
| Challenge: | Existing semantic parsing datasets lack a single standard for meaning representations . lack of a standard led to the creation of plethora of datasets requiring expert annotators . |
| Approach: | They propose to use multi-task learning to unify different datasets and train a single model for them. |
| Outcome: | The proposed architectures yield better parsing accuracies and composition generalization than single-task models. |
Functional Distributional Semantics at Scale (2023.starsem-1)
Copied to clipboard
| Challenge: | Functional Distributional Semantics is a linguistically motivated framework for modelling lexical and sentence-level semantics with truth-conditional functions using distributional information. |
| Approach: | They propose a more expressive lexical model that works over a continuous semantic space. |
| Outcome: | The proposed model improves performance and flexibility and is compatible with present-day machine learning frameworks. |
Inducing Language-Agnostic Multilingual Representations (2021.starsem-1)
Copied to clipboard
| Challenge: | Cross-lingual representations have the potential to make NLP techniques available to the vast majority of languages in the world, but they currently require large pretraining corpora or access to typologically similar languages. |
| Approach: | They propose to remove language identity signals from multilingual embeddings by re-aligning vector spaces of target languages to a pivot source language and removing language-specific means and variances. |
| Outcome: | The proposed approaches reduce cross-lingual transfer gap by 8.9 points (m-BERT) and 18.2 points (XLM-R) on average across all tasks and languages. |
Exploring Factual Entailment with NLI: A News Media Study (2024.starsem-1)
Copied to clipboard
| Challenge: | Recent studies have focused on the relationship between factuality and Natural Language Inference (NLI). |
| Approach: | They propose a novel annotation scheme that models factual rather than textual entailment and use it to annotate a dataset of naturally occurring sentences from news articles. |
| Outcome: | The proposed annotation scheme can be used to model factual relationships on a dataset of naturally occurring sentences from news articles. |
Empirical Sufficiency Lower Bounds for Language Modeling with Locally-Bootstrapped Semantic Structures (2023.starsem-1)
Copied to clipboard
| Challenge: | a recent attempt at language modeling with predicted semantic structure failed to establish empirical lower bounds on what could have made the attempt successful. |
| Approach: | They propose a concise binary vector representation of semantic structure at the lexical level and evaluate how good an incremental tagger needs to be to achieve better-than-baseline performance. |
| Outcome: | The proposed model can achieve better-than-baseline performance without losing its main advantages and lower bounds on prediction quality can't be established via a single score alone. |