It is a Bird Therefore it is a Robin: On BERT’s Internal Consistency Between Hypernym Knowledge and Logical Words (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing models for lexical knowledge of NLP systems only use collocational cues for questions . lexicon of these systems is limited to content words and logical words. |
| Approach: | They propose a method to test the understanding of the hypernymy relationship by measuring its antisymmetry according to the models. |
| Outcome: | The proposed model shows that it is weak on the semantic tasks of the Masked Language Modeling task. |
Similar Papers
Does BERT Know that the IS-A Relation Is Transitive? (2022.acl-short)
Copied to clipboard
| Challenge: | Recent studies suggest pre-trained BERT can capture lexico-semantic clues from words in context. |
| Approach: | They examine word senses and the transitive property of IS-A relation . they aim to quantify how much BERT agrees with transitivity property . |
| Outcome: | The proposed model can capture lexico-semantic clues from words in context . but to what extent it captures transitive nature of some lexical relations is unclear . |
What BERT Is Not: Lessons from a New Suite of Psycholinguistic Diagnostics for Language Models (2020.tacl-1)
Copied to clipboard
| Challenge: | Pretraining by language modeling has become popular but we have yet to understand what language models learn during that process. |
| Approach: | They propose diagnostics that ask questions about information used by language models for generating predictions in context. |
| Outcome: | The proposed diagnostics can be used to study the popular BERT model . they show that the model can distinguish good from bad completions, but struggles with inference and role-based event prediction. |
Language Models and Semantic Relations: A Dual Relationship (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies on language models for the extraction of semantic relations have focused on injecting semantic knowledge into these models to enhance them. |
| Approach: | They propose to extract lexical semantic relations from a BERT model and inject them into it using unsupervised methods based on semantic similarity at word and sentence levels. |
| Outcome: | The proposed method allows to enrich a BERT model without using any external semantic resource. |
Specialising Word Vectors for Lexical Entailment (N18-1)
Copied to clipboard
| Challenge: | Existing word representation learning methods rely on the distributional hypothesis to learn meaningful word representations. |
| Approach: | They propose a method that emphasises the asymmetric relation of lexical entailment by injecting external linguistic constraints into the input word vector space. |
| Outcome: | The proposed method achieves state-of-the-art in the tasks of hypernymy directionality, hypernomia detection, and graded lexical entailment. |
When Hearst Is not Enough: Improving Hypernymy Detection from Corpus with Distributional Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | a taxonomy is a semantic hierarchy of words or concepts organized w.r.t. their hypernymy relationships. |
| Approach: | They propose a framework for hypernymy detection using large textual corpora . they quantify the non-negligible existence of specific sparsity cases . |
| Outcome: | The proposed framework quantifies the non-negligible existence of specific sparsity cases on several benchmark datasets. |
Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference (P19-1)
Copied to clipboard
| Challenge: | lexical overlap heuristics are effective for frequent example types but break down in more challenging cases. |
| Approach: | They propose to use a set of examples to test whether a sentence entails another . they propose to adopt three fallible syntactic heuristics for statistical NLI models . |
| Outcome: | The proposed model performs poorly on the HANS dataset, suggesting they have adopted these heuristics. |
Using Paraphrases to Study Properties of Contextual Embeddings (2022.naacl-main)
Copied to clipboard
| Challenge: | Previously, paraphrases have been used to probe whether compositionality is accurately captured by BERT, but we believe they can be used to explore many other questions. |
| Approach: | They propose to use paraphrases as a unique source of data to analyze contextualized embeddings, with a particular focus on BERT. |
| Outcome: | The proposed analysis of paraphrases and paraphrase representations using the Paraphrase Database shows that BERT handles polysemous words, but different representations in many cases. |
He Thinks He Knows Better than the Doctors: BERT for Event Factuality Fails on Pragmatics (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing models for factuality prediction are lacking for English . Traditionally, event factualism is triggered by fixed properties of lexical items . |
| Approach: | They propose a model that exploits common surface patterns that correlate with factuality labels. |
| Outcome: | The proposed model achieves the best performance on four factuality datasets. |
Let’s Play Mono-Poly: BERT Can Reveal Words’ Polysemy Level and Partitionability into Senses (2021.tacl-1)
Copied to clipboard
| Challenge: | Pre-trained language models encode rich information about linguistic structure but their knowledge about lexical polysemy remains unclear. |
| Approach: | They propose a setup for analyzing lexical polysemy knowledge in pre-trained language models and multilingual BERT models by analyzing different sense distributions and controlling for parameters that are highly correlated with polysyntax. |
| Outcome: | The proposed model can be used to analyze lexical polysemy in English, French, Spanish, and Greek and in multilingual BERT. |
Evaluating BERT for natural language inference: A case study on the CommitmentBank (D19-1)
Copied to clipboard
| Challenge: | Natural language inference datasets can identify premise-hypothesis relationship without observing premise . recasting of the CommitmentBank for NLI creates hypotheses that stand in entailment/contradiction/neutral relationship with premise. |
| Approach: | They propose to recast the CommitmentBank for NLI to stand in certain relationships with the premise . hypotheses are complements of clause-embedding verbs in each premise, rethinking the CommittedBank . |
| Outcome: | The proposed model performs well on the CommitmentBank with 85% F1 . however, the model does not capture the full complexity of pragmatic reasoning, authors say . |