| Challenge: | Unsupervised pre-trained word embeddings are used for many tasks in natural language processing to leverage unlabeled textual data. |
| Approach: | They extend the model's task loss with an unsupervised auxiliary loss on the word-embedding level of the model to ensure that the learned word representations contain both task-specific features and more general features. |
| Outcome: | The proposed model improves on the task of extracting narrative containment relations from clinical records using a general-domain part-of-speech tagger as linguistic resource. |
Similar Papers
Exploiting Entity BIO Tag Embeddings and Multi-task Learning for Relation Extraction with Imbalanced Data (P19-1)
Copied to clipboard
| Challenge: | Existing methods to perform relation extraction are feature-based or kernel-based, but the results of our study show that they can improve the performance of a baseline model with more than 10% absolute increase in F1-score. |
| Approach: | They propose a multi-task architecture which jointly trains a model to perform relation identification with cross-entropy loss and relation classification with ranking loss. |
| Outcome: | The proposed model outperforms the state-of-the-art models on ACE 2005 Chinese and English corpus and significantly improves the performance of a baseline model with more than 10% increase in F1-score. |
Entity or Relation Embeddings? An Analysis of Encoding Strategies for Relation Extraction (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to relation extraction use concatenating embeddings of head and tail entities . however, such representations capture the types of the entities involved, leading to false positives and confusion between relations involving entities of the same type. |
| Approach: | They propose a model which combines [MASK] embeddings with entity embedds to learn relation embeddations. |
| Outcome: | The proposed model outperforms the state-of-the-art on several benchmarks . it uses a self-supervised pre-training strategy which further improves the results. |
Cross-lingual Sentence Embedding using Multi-Task Learning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing multilingual sentence embedding models require large parallel corpora to learn efficiently, limiting their scope. |
| Approach: | They propose a sentence embedding framework based on an unsupervised loss function . they capture semantic similarity and relatedness between sentences using a multi-task loss function. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on STS, BUCC and Tatoeba benchmarks and on a monolingual benchmark. |
Unsupervised Information Extraction: Regularizing Discriminative Approaches with Relation Distribution Losses (P19-1)
Copied to clipboard
| Challenge: | Existing unsupervised relation extraction models are either generative or discriminative . however, they are hard to train without supervision and are unstable . |
| Approach: | They propose a skewness loss and distribution distance loss to improve the performance of discriminative based models. |
| Outcome: | The proposed models surpass current state-of-the-art on three different datasets. |
Incorporating Global Contexts into Sentence Embedding for Relational Extraction at the Paragraph Level with Distant Supervision (L18-1)
Copied to clipboard
| Challenge: | Existing approaches to relation extraction (RE) only extract relations from sentences that contain two target entities. |
| Approach: | They propose to incorporate global contexts from paragraph-into-sentence embedding into RE . they propose to use a knowledge base to extract relations between pairs of entities . |
| Outcome: | The proposed approach can learn an exact RE from sentences without syntactic parsing. |
An Improved Neural Baseline for Temporal Relation Extraction (D19-1)
Copied to clipboard
| Challenge: | Existing datasets are small and/or have low inter-annotator agreements. |
| Approach: | They propose a new neural system that achieves 10% absolute accuracy improvement over the previous best system. |
| Outcome: | The proposed system achieves 10% absolute improvement over the previous best system on two benchmark datasets. |
Global Relation Embedding for Relation Extraction (N18-1)
Copied to clipboard
| Challenge: | Existing methods to extract textual relations with distant supervision are limited by their reliance on supervised training data. |
| Approach: | They propose to embed relations with global statistics of relations to combat the wrong labeling problem of distant supervision. |
| Outcome: | The proposed method is more robust to training noise introduced by distant supervision and improves relation extraction models. |
HiURE: Hierarchical Exemplar Contrastive Learning for Unsupervised Relation Extraction (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to extract relational feature signals from natural language sentences use self-supervised clustering and classification that cause gradual drift problems. |
| Approach: | They propose a framework that derives hierarchical signals from relational feature space using cross hierarchy attention and effectively optimizes relation representation of sentences under exemplar-wise contrastive learning. |
| Outcome: | The proposed framework can extract the relationship between entities from natural language sentences without prior knowledge on relation scope or distribution. |
Unsupervised Relation Extraction: A Variational Autoencoder Approach (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for relation extraction use latent variables and supervised training which requires large datasets. |
| Approach: | They propose a VAE-based unsupervised relation extraction technique that uses latent variables as an intermediate variable instead of a latent variable. |
| Outcome: | The proposed method outperforms state-of-the-art methods on the NYT dataset and outperformed existing methods. |
DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations (2021.acl-long)
Copied to clipboard
| Challenge: | Sentence embeddings are an important component of many natural language processing systems. |
| Approach: | They propose a self-supervised objective for learning universal sentence embeddings that does not require labelled training data. |
| Outcome: | The proposed approach closes the performance gap between unsupervised and supervised pretraining for universal sentence encoders. |