Don’t Invite BERT to Drink a Bottle: Modeling the Interpretation of Metonymies Using BERT and Distributional Representations (2020.coling-main)
Copied to clipboard
| Challenge: | a recent study has shown that metonymy is a productive and systematic process . linguistic and psycholinguistic studies support the idea that metnomic interpretations are based on lexical ambiguity . |
| Approach: | They compare BERT to a generalized event knowledge model to capture the meaning shift associated with metonymy. |
| Outcome: | The proposed model is good at predicting the meaning of metonymic expressions, the authors say . they show that the model can capture the meaning shift associated with metonymy . |
Similar Papers
What BERT Is Not: Lessons from a New Suite of Psycholinguistic Diagnostics for Language Models (2020.tacl-1)
Copied to clipboard
| Challenge: | Pretraining by language modeling has become popular but we have yet to understand what language models learn during that process. |
| Approach: | They propose diagnostics that ask questions about information used by language models for generating predictions in context. |
| Outcome: | The proposed diagnostics can be used to study the popular BERT model . they show that the model can distinguish good from bad completions, but struggles with inference and role-based event prediction. |
ConMeC: A Dataset for Metonymy Resolution with Common Nouns (2025.naacl-long)
Copied to clipboard
| Challenge: | Prior work on metonymy resolution has focused on named entities, but common nouns are also a frequent problem. |
| Approach: | They propose a dataset that combines a metonymy dataset and a chain-of-thought based prompting method for detecting metonyms using large language models. |
| Outcome: | The proposed method can detect metonymy using large language models while still struggling with nuanced semantic understanding. |
A Primer in BERTology: What We Know About How BERT Works (2020.tacl-1)
Copied to clipboard
| Challenge: | a new study examines the current state of knowledge about the BERT model . the model is a stack of transformer encoder layers that are based on multiple self-attention ''heads'' |
| Approach: | They present a survey of over 150 studies of the popular Transformer-based model BERT . they discuss the current state of knowledge about how BERT works and how it is represented . |
| Outcome: | The proposed model is based on the Transformer-based model with state-of-the-art results . the proposed model has little cognitive motivation and is too small to perform ablation studies . |
From BERT‘s Point of View: Revealing the Prevailing Contextual Differences (2022.findings-acl)
Copied to clipboard
| Challenge: | BERTology is a new approach to understanding the inner workings of large pretraining language models. |
| Approach: | They propose to invert the probing design to analyze the prevailing differences and clusters in BERT’s high dimensional space by extracting coarse features from masked token representations and predicting them by probing models with access to only partial information. |
| Outcome: | The proposed method extracts coarse features from masked token representations and predicts them by probing models with access to only partial information. |
Does BERT Recognize an Agent? Modeling Dowty’s Proto-Roles with Contextual Embeddings (2022.coling-1)
Copied to clipboard
| Challenge: | Contextual embeddings build multidimensional representations of word tokens based on their context of occurrence. |
| Approach: | They propose to map the verb embeddings to an interpretable space of semantic properties built from a linguistic dataset and test their ability to model the semantic properties of the agent of the verbs participating in the alternation. |
| Outcome: | The proposed models can model the semantic properties of the verbs participating in the so-called causative alternation. |
Probe-Less Probing of BERT’s Layer-Wise Linguistic Knowledge with Masked Word Prediction (2022.naacl-srw)
Copied to clipboard
| Challenge: | Among studies on localization of linguistic knowledge, it is unclear what information is encoded in each layer. |
| Approach: | They analyze BERT’s layer-wise masked word prediction on an English corpus and find syntactic and semantic information is encoded at different layers for words of different syntaktic categories. |
| Outcome: | The proposed model outperforms state-of-the-art models in many downstream tasks. |
Why is penguin more similar to polar bear than to sea gull? Analyzing conceptual knowledge in distributional models (2020.acl-srw)
Copied to clipboard
| Challenge: | Several analysis methods have been shown to be limited and are not well understood . thesis aims to understand distributional semantic representations based on linguistic data . |
| Approach: | They propose a framework for investigating the information encoded in distributional semantic models . they combine observations made on corpora with insights obtained from data manipulation experiments . |
| Outcome: | The proposed framework pairs observations made on corpora with insights obtained from data manipulation experiments. |
Language Models and Semantic Relations: A Dual Relationship (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies on language models for the extraction of semantic relations have focused on injecting semantic knowledge into these models to enhance them. |
| Approach: | They propose to extract lexical semantic relations from a BERT model and inject them into it using unsupervised methods based on semantic similarity at word and sentence levels. |
| Outcome: | The proposed method allows to enrich a BERT model without using any external semantic resource. |
Does BERT Know that the IS-A Relation Is Transitive? (2022.acl-short)
Copied to clipboard
| Challenge: | Recent studies suggest pre-trained BERT can capture lexico-semantic clues from words in context. |
| Approach: | They examine word senses and the transitive property of IS-A relation . they aim to quantify how much BERT agrees with transitivity property . |
| Outcome: | The proposed model can capture lexico-semantic clues from words in context . but to what extent it captures transitive nature of some lexical relations is unclear . |
He Thinks He Knows Better than the Doctors: BERT for Event Factuality Fails on Pragmatics (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing models for factuality prediction are lacking for English . Traditionally, event factualism is triggered by fixed properties of lexical items . |
| Approach: | They propose a model that exploits common surface patterns that correlate with factuality labels. |
| Outcome: | The proposed model achieves the best performance on four factuality datasets. |