Did the Cat Drink the Coffee? Challenging Transformers with Generalized Event Knowledge (2021.starsem-1)
Copied to clipboard
Paolo Pedinotti, Giulia Rambelli, Emmanuele Chersoni, Enrico Santus, Alessandro Lenci, Philippe Blache
| Challenge: | Prior work has explored the ability of computational models to predict word semantic fit with a given predicate. |
| Approach: | They compare Transformers Language Models to SDM to assess their performance . they found that TLMs do not capture important aspects of event knowledge . people can discriminate between typical and atypical events, they say . |
| Outcome: | The proposed models can achieve comparable performance to SDM, but they lack important aspects of event knowledge. |
Similar Papers
Event Semantic Knowledge in Procedural Text Understanding (2023.starsem-1)
Copied to clipboard
| Challenge: | Annotators’ reliance on commonsense knowledge to annotate implicit state information is a challenge for entity state tracking. |
| Approach: | They propose a method for entity state tracking that incorporates commonsense entity-centric knowledge from ConceptNet into a BERT-based neural-symbolic architecture. |
| Outcome: | The proposed model outperforms existing models on the ProPara dataset and is domain-agnostic. |
Capturing the Content of a Document through Complex Event Identification (2022.starsem-1)
Copied to clipboard
| Challenge: | Recent work grouped granular events into more general events, called complex events . however, this approach assumes that a given complex event is always described in consecutive sentences . |
| Approach: | They propose a context-augmented representation learning approach that uses contextual information to model pairwise relation between granular events. |
| Outcome: | The proposed approach outperforms baselines on the complex event identification task. |
Syntax and Semantics Meet in the “Middle”: Probing the Syntax-Semantics Interface of LMs Through Agentivity (2023.starsem-1)
Copied to clipboard
| Challenge: | a recent study examined how large language models handle interactions in meaning across words and larger syntactic forms. |
| Approach: | They propose to use a dataset to examine the linguistic properties of optionally transitive English verbs to examine their agentivity. |
| Outcome: | The proposed model outperforms all other models in the evaluation dataset . the results are better correlated with human judgements than syntactic and semantic corpus statistics . |
Investigating Aspect Features in Contextualized Embeddings with Semantic Scales and Distributional Similarity (2024.starsem-1)
Copied to clipboard
| Challenge: | Aspect is a linguistic category describing how actions and events unfold over time. |
| Approach: | They propose to use semantic projections to examine whether the vector dimensions of annotated verbs reflect human linguistic distinctions. |
| Outcome: | The proposed models encode the aspects of stativity, durativity and telicity in most of their layers, while durativité is the most challenging feature. |
Limits for learning with language models (2023.starsem-1)
Copied to clipboard
| Challenge: | Recent studies show that large language models fail to capture important aspects of linguistic meaning . authors argue that LLMs cannot learn fundamental semantic properties defined in formal semantics . |
| Approach: | They propose a theoretical explanation for some of the observed failings of large language models . they show that LLMs cannot learn certain fundamental semantic properties . |
| Outcome: | The proposed model fails to learn semantic entailment and consistency as defined in formal semantics, the authors argue . their model fails on tasks that require engorgements and deep linguistic understanding, they argue - but not on universal quantification. |
Comparison and Combination of Sentence Embeddings Derived from Different Supervision Signals (2022.starsem-1)
Copied to clipboard
| Challenge: | Existing methods to derive sentence embeddings have not been well understood what properties are captured in the resulting sentences depending on the supervision signals. |
| Approach: | They propose to combine two types of sentence embedding methods with similar architectures and tasks to investigate their properties. |
| Outcome: | The proposed methods perform better on unsupervised and downstream tasks than the proposed methods on untrained STS tasks and probing tasks. |
What do Large Language Models Learn about Scripts? (2022.starsem-1)
Copied to clipboard
| Challenge: | Script Knowledge is important for language understanding but expensive to produce manually and difficult to induce from text due to reporting bias. |
| Approach: | They propose a pipeline-based script induction framework which can generate good quality ESDs for unseen scenarios. |
| Outcome: | The proposed framework produces good quality ESDs for unseen scenarios, but manual evaluation shows there is room for improvement. |
Lexical Substitution as Causal Language Modeling (2024.starsem-1)
Copied to clipboard
| Challenge: | Existing methods for lexical substitution task lacks autoregressive decoding capabilities. |
| Approach: | They propose a framework that uses causal language modeling (CLM) for lexical substitution task. |
| Outcome: | The proposed system outperforms GeneSis, the best previously published supervised LST method. |
Disambiguating Emotional Connotations of Words Using Contextualized Word Representations (2024.starsem-1)
Copied to clipboard
| Challenge: | BERT, RoBERTa, XLNet, and GPT-2 models effectively discern emotional connotations of words, demonstrating superior performance and greater resilience against biases. |
| Approach: | They propose to use contextualized word representations to examine how words can be used to distinguish emotional connotations across contexts. |
| Outcome: | The proposed models show that they can distinguish emotional connotations of words in different contexts. |
Polysemy through the lens of psycholinguistic variables: a dataset and an evaluation of static and contextualized language models (2024.starsem-1)
Copied to clipboard
| Challenge: | Polysemes are words that can have different senses depending on context . traditionally, NLP models assume that each sense should be given a separate representation in a lexicon, thus limiting the amount of evidence that can be gained from their use. |
| Approach: | They propose a framework to model polysemes as a continuous variation in psycholinguistic properties of a word in context without postulating jumps between senses. |
| Outcome: | The proposed framework accommodates different sense interpretations, without postulating clear-cut jumps between senses. |