Classification and Clustering of Arguments with Contextualized Word Embeddings (P19-1)
Copied to clipboard
| Challenge: | Existing methods for argument mining focus on analyzing local argumentation structures, but information-seeking approaches need to be able to deal with heterogeneous sources and topics. |
| Approach: | They propose to use contextualized word embeddings to classify and cluster topic-dependent arguments using a UKP Sentential Argument Mining Corpus and IBM Debater - Evidence Sentences datasets. |
| Outcome: | The proposed method improves state-of-the-art on argument classification and clustering tasks and across multiple datasets. |
Similar Papers
Contextualized Embeddings for Enriching Linguistic Analyses on Politeness (2020.coling-main)
Copied to clipboard
| Challenge: | Current word embeddings in natural language processing do capture context and thus can be leveraged to enrich linguistic analyses. |
| Approach: | They propose a model which leverages pre-trained BERT to cluster contextualized representations of a word based on context in which it appears and labels of items it occurs in. |
| Outcome: | The proposed model can detect interpretable, finer-grained context patterns associated with (im)polite language. |
Using Paraphrases to Study Properties of Contextual Embeddings (2022.naacl-main)
Copied to clipboard
| Challenge: | Previously, paraphrases have been used to probe whether compositionality is accurately captured by BERT, but we believe they can be used to explore many other questions. |
| Approach: | They propose to use paraphrases as a unique source of data to analyze contextualized embeddings, with a particular focus on BERT. |
| Outcome: | The proposed analysis of paraphrases and paraphrase representations using the Paraphrase Database shows that BERT handles polysemous words, but different representations in many cases. |
Obtaining Better Static Word Embeddings Using Contextual Embedding Models (2021.acl-long)
Copied to clipboard
| Challenge: | Recent contextual word embeddings have prohibitively high computational cost in many use-cases and are hard to interpret. |
| Approach: | They propose a distillation method which is an extension of CBOW-based training and improves computational efficiency of NLP applications. |
| Outcome: | The proposed method outperforms existing models and existing models in terms of quality and performance. |
Limited Generalizability in Argument Mining: State-Of-The-Art Models Learn Datasets, Not Arguments (2025.acl-long)
Copied to clipboard
| Challenge: | Identifying arguments is a prerequisite for various tasks in automated discourse analysis. |
| Approach: | They evaluate four BERT-like transformers on 17 English sentence-level datasets . they find that they tend to rely on lexical shortcuts tied to content words . |
| Outcome: | The proposed models perform best on 17 English sentence-level datasets on common tasks, but their performance drops when applied to unseen datasets. |
Contextual Embeddings: When Are They Worth It? (2020.acl-main)
Copied to clipboard
| Challenge: | In recent years, rich contextual embeddings have enabled rapid progress on benchmarks like GLUE, but require significant computational resources during pretraining and during downstream task training and inference. |
| Approach: | They empirically compare contextual embeddings with classic pretrained embedders and a random word embeddable with a simple baseline. |
| Outcome: | The proposed models perform within 5 to 10% accuracy on industry-scale data. |
Contextual Argument Component Classification for Class Discussions (2020.coling-main)
Copied to clipboard
| Challenge: | Argument mining systems often consider contextual information when training to perform tasks such as argument component identification, classification, and relation extraction. |
| Approach: | They propose to incorporate speaker context and local discourse context into a model for classifying argument components in multi-party classroom discussions. |
| Outcome: | The proposed model improves when varying context size and position . the results support the claim that context size is important . |
Building Static Embeddings from Contextual Ones: Is It Useful for Building Distributional Thesauri? (2022.lrec-1)
Copied to clipboard
| Challenge: | contextual language models are dominant in the field of Natural Language Processing, but they are not suitable for all uses. |
| Approach: | They propose a method for building word or type-level embeddings from contextual models . they evaluate a large set of English nouns from the perspective of extracting semantic similarity relations . |
| Outcome: | The proposed method can be used to build word or type embeddings from contextual models . it can be exploited for a wide set of English nouns, showing it can improve distributional thesauri . |
Verb Sense Clustering using Contextualized Word Representations for Semantic Frame Induction (2021.findings-acl)
Copied to clipboard
| Challenge: | Contextualized word representations are effective in many natural language processing tasks, but it remains unclear to what extent they can cover hand-coded semantic information such as semantic frames. |
| Approach: | They compare contextualized word representations with two English frame-semantic resources . they find that several contextualized representations are informative for semantic frame induction . |
| Outcome: | The proposed representations are useful in natural language processing tasks, but are not fully understood by the literature. |
Embeddings in Natural Language Processing (2020.coling-tutorials)
Copied to clipboard
| Challenge: | Embeddings have been a key topic of interest in NLP for the past decade . a quick warm-up introduction to NLP and why it is important to have a semantic comprehension of texts . |
| Approach: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and then move to other types of embeddable vectors . |
| Outcome: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and move to other types of embeddable representations . |
Retrofitting Contextualized Word Embeddings with Paraphrases (D19-1)
Copied to clipboard
| Challenge: | Contextualized word embeddings can be useful for downstream applications, but they can be over-sensitive to contexts. |
| Approach: | They propose a method to retrofit contextualized word embeddings with paraphrases to minimize the variance of word representations on paraphrased contexts. |
| Outcome: | The proposed method improves on sentence classification and inference tasks. |