Multi-Level Structured Self-Attentions for Distantly Supervised Relation Extraction (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to label large-scale data are inadequate for distantly supervised relation extraction (DS-RE). |
| Approach: | They propose a multi-level structured (2-D matrix) self-attention mechanism for DS-RE using bidirectional recurrent neural networks. |
| Outcome: | The proposed framework significantly outperforms baselines on two publicly available DS-RE datasets in terms of PR curves, P@N and F1 measures. |
Similar Papers
Hierarchical Relation Extraction with Coarse-to-Fine Grained Attention (D18-1)
Copied to clipboard
| Challenge: | Existing methods for relation extraction use knowledge graphs to automatically label training data . but, it suffers from the wrong labeling problem because not all sentences containing two entities can express their relations in KGs . |
| Approach: | They propose a distant supervision approach to automatically label training instances . they integrate hierarchical information of relations into distantly supervised relation extraction . |
| Outcome: | The proposed model outperforms baseline models on a large-scale dataset. |
Modular Self-Supervision for Document-Level Relation Extraction (2021.emnlp-main)
Copied to clipboard
| Challenge: | Prior work on information extraction tends to focus on binary relations within sentences . practical applications often require extracting complex relations across large text spans . |
| Approach: | They propose to decompose document-level relation extraction into relation detection and argument resolution, taking inspiration from Davidsonian semantics. |
| Outcome: | The proposed method outperforms state-of-the-art methods in biomedical machine reading for precision oncology by 20 absolute F1 points. |
Phrase-level Self-Attention Networks for Universal Sentence Encoding (D18-1)
Copied to clipboard
| Challenge: | Phrase-level self-attention networks (PSAN) can capture context dependencies at the phrase level instead of the sentence level. |
| Approach: | They propose to perform self-attention across words inside a phrase to capture context dependencies at the phrase level and use the gated memory updating mechanism to refine each word’s representation hierarchically with longer-term context dependency captured in a larger phrase. |
| Outcome: | The proposed model can achieve state-of-the-art performance across a plethora of NLP tasks including binary and multi-class classification, natural language inference and sentence similarity. |
Progressive Self-Supervised Attention Learning for Aspect-Level Sentiment Analysis (P19-1)
Copied to clipboard
| Challenge: | Experimental results show that our proposed approach yields better attention mechanisms . dominant ASC models are mostly discriminative classifiers based on manual feature engineering . |
| Approach: | They propose a self-supervised approach to aspect-level sentiment classification that mines useful attention supervision information from a training corpus to refine attention mechanisms. |
| Outcome: | The proposed approach yields better attention mechanisms on multiple datasets. |
HiCLRE: A Hierarchical Contrastive Learning Framework for Distantly Supervised Relation Extraction (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches of distantly supervised relation extraction (DSRE) focus on sentence-level or bag-level de-noising, neglecting the explicit interaction with cross levels. |
| Approach: | They propose a hierarchical contrastive learning framework for distantly supervised relation extraction to reduce noisy sentences. |
| Outcome: | The proposed framework outperforms baselines in various mainstream DSRE datasets. |
Recurrent Attention Networks for Long-text Modeling (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to encoding long documents using self-attention have been limited by quadratic computational complexities and limited application in long text processing. |
| Approach: | They propose a long-document encoding model that allows the recurrent operation of self-attention. |
| Outcome: | The proposed model extracts global semantics in token-level and document-level representations, making it inherently compatible with both sequential and sequential tasks. |
Improving Relation Extraction with Knowledge-attention (D19-1)
Copied to clipboard
| Challenge: | Existing attention mechanisms are data-driven, but most are data driven. |
| Approach: | They propose a knowledge-attention encoder which integrates prior knowledge from external lexical resources into deep neural networks for relation extraction task. |
| Outcome: | The proposed system outperforms existing CNN, RNN, and self-attention based models on a large-scale relation extraction dataset. |
Self-Attention Enhanced CNNs and Collaborative Curriculum Learning for Distantly Supervised Relation Extraction (D19-1)
Copied to clipboard
| Challenge: | Distantly Supervised Relation Extraction (DSRE) suffers from mislabelled data . human annotation on large datasets is costly and often impossible . |
| Approach: | They propose a model that employs a collaborative curriculum learning framework to reduce mislabelled data. |
| Outcome: | The proposed model outperforms baselines including state-of-the-art in terms of P@N and PR curve metrics on a widely-used public dataset. |
Convolutional Self-Attention Networks (N19-1)
Copied to clipboard
| Challenge: | Existing models of self-attention networks lack the ability to capture dependencies regardless of distance and can be enhanced with multi-head attention. |
| Approach: | They propose a convolutional self-attention network which can be enhanced by multi-head attention by allowing the model to attend to information from different representation subspaces. |
| Outcome: | The proposed model outperforms existing models on improving locality of SANs on different language pairs and model settings. |
A Novel Perspective to Look At Attention: Bi-level Attention-based Explainable Topic Modeling for News Classification (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing deep learning models have the attention mechanism to improve performance, but the inherent characteristics of deep learning model complexity and the flexibility of the attention structure make them difficult to explain. |
| Approach: | They propose a two-tier attention architecture to decouple the complexity of explanation and the decision-making process by using large-scale news corpora. |
| Outcome: | The proposed model can achieve competitive performance with state-of-the-art models and illustrates its appropriateness from an explainability perspective. |