Token-level Dynamic Self-Attention Network for Multi-Passage Reading Comprehension (P19-1)
Copied to clipboard
| Challenge: | Multi-passage reading comprehension requires the ability to combine cross-passages information and reason over multiple passages to infer the answer. |
| Approach: | They propose a Dynamic Self-attention Network (DynSAN) which processes cross-passage information at token-level and meanwhile avoids substantial computational costs. |
| Outcome: | The proposed model achieves state-of-the-art performance on the SearchQA, Quasar-T and WikiHop datasets and further ablation validates the effectiveness of its components. |
Similar Papers
Phrase-level Self-Attention Networks for Universal Sentence Encoding (D18-1)
Copied to clipboard
| Challenge: | Phrase-level self-attention networks (PSAN) can capture context dependencies at the phrase level instead of the sentence level. |
| Approach: | They propose to perform self-attention across words inside a phrase to capture context dependencies at the phrase level and use the gated memory updating mechanism to refine each word’s representation hierarchically with longer-term context dependency captured in a larger phrase. |
| Outcome: | The proposed model can achieve state-of-the-art performance across a plethora of NLP tasks including binary and multi-class classification, natural language inference and sentence similarity. |
Leveraging Local and Global Patterns for Self-Attention Networks (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to integrate local and global information into self-attention networks have been criticized for overlooking neighboring information. |
| Approach: | They propose a hybrid attention mechanism to leverage local and global information . they use a gating scalar to integrate both sources of information based on local contexts . |
| Outcome: | The proposed approach improves on translation tasks and shows that the two types of contexts are complementary. |
A Multi-Type Multi-Span Network for Reading Comprehension that Requires Discrete Reasoning (D19-1)
Copied to clipboard
| Challenge: | Existing models for reading comprehension and question answering do not support discrete reasoning abilities. |
| Approach: | They propose a reading comprehension model that uses a multi-type answer predictor and a multiple-span extraction method to produce one or multiple text spans. |
| Outcome: | The proposed model achieves 79.9 F1 on the DROP hidden test set, creating new state-of-the-art results. |
Multi-Level Structured Self-Attentions for Distantly Supervised Relation Extraction (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to label large-scale data are inadequate for distantly supervised relation extraction (DS-RE). |
| Approach: | They propose a multi-level structured (2-D matrix) self-attention mechanism for DS-RE using bidirectional recurrent neural networks. |
| Outcome: | The proposed framework significantly outperforms baselines on two publicly available DS-RE datasets in terms of PR curves, P@N and F1 measures. |
Read and Comprehend by Gated-Attention Reader with More Belief (N18-4)
Copied to clipboard
| Challenge: | Existing approaches to read comprehension using gated-attention have been effective . collaborative gating and self-belief aggregation are proposed to address these assumptions . |
| Approach: | They propose to use a document-to-query attention system to gate token encodings of a query . they conjecture that query tokens other than the cloze token may be informative . |
| Outcome: | The proposed approaches advance the state-of-the-art results in CNN, Daily Mail, and Who Did What public test sets. |
Multi-Granularity Hierarchical Attention Fusion Networks for Reading Comprehension and Question Answering (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to read comprehension style question answering are limited by the volume of annotated datasets. |
| Approach: | They propose a hierarchical attention network for reading comprehension style question answering . they first encode the question and paragraph with fine-grained language embeddings . then propose fusion approach to fuse information from both global and attended representations based on the hierarchic attention network . |
| Outcome: | The proposed method achieves state-of-the-art on the SQuAD and TriviaQA Wiki leaderboards and two adversarial SQu AD datasets. |
Document Modeling with Graph Attention Networks for Multi-grained Machine Reading Comprehension (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to machine reading comprehension treat documents at their hierarchical nature, ignoring their dependencies. |
| Approach: | They propose a machine reading comprehension benchmark with two-grained answers . they use graph attention networks to model documents at their hierarchical nature . |
| Outcome: | The proposed framework outperforms existing systems at long and short answer criteria. |
Dynamic Feature Selection with Attention in Incremental Parsing (C18-1)
Copied to clipboard
| Challenge: | Currently, incremental transition-based parsers require that all inputs are visible from the beginning to extract good features from a limited local context. |
| Approach: | They propose a technique to maximize local features with an attention mechanism which works as context- dependent dynamic feature selection. |
| Outcome: | The proposed technique can extract features from a limited local context and is able to perform multilingual experiments and demon strate on local ambiguous points. |
Reading Turn by Turn: Hierarchical Attention Architecture for Spoken Dialogue Comprehension (P19-1)
Copied to clipboard
| Challenge: | Existing research on multi-turn spoken conversations focuses on reading comprehension of passages . interactivity of spoken content can cause lower information density and topic diffusion . |
| Approach: | They propose a hierarchical attention neural network architecture to improve spoken dialogue comprehension by combining turn-level and word-level attention mechanisms. |
| Outcome: | The proposed approach outperforms baseline attention models and is robust to lengthy and out-of-distribution test samples. |
Cut to the Chase: A Context Zoom-in Network for Reading Comprehension (D18-1)
Copied to clipboard
| Challenge: | Recent deep-learning based models suffer from reasoning over long documents and do not trivially generalize to cases where the answer is not present as a span. |
| Approach: | They propose a novel context zoom-in network (ConZNet) that can skip through irrelevant parts of a document and generate an answer using only the relevant regions of text. |
| Outcome: | The proposed architecture outperforms state-of-the-art results by 12.62% (ROUGE-L) relative improvement on the recently proposed and challenging RC dataset ‘NarrativeQA’. |