| Challenge: | Existing methods for sentence ordering tasks rely on linguistic knowledge and are domain specific. |
| Approach: | They propose a deep attentive sentence ordering network which integrates self-attention mechanism with LSTMs in the encoding of input sentences. |
| Outcome: | The proposed model outperforms the state-of-the-art models on Sentence Ordering and Order Discrimination tasks and is shown to be highly efficient. |
Similar Papers
Higher-Order Syntactic Attention Network for Longer Sentence Compression (N18-1)
Copied to clipboard
| Challenge: | Existing sentence compression methods do not handle syntactic features, causing performance degradation . et al. (2015) reported that the longer the input sentences are, the worse the performance becomes. |
| Approach: | They propose a higher-order syntactic attention network that handles higher-level dependency features as an attention distribution on LSTM hidden states. |
| Outcome: | The proposed method outperforms baseline methods on a Google sentence compression dataset. |
Towards Better Modeling Hierarchical Structure for Self-Attention with Ordered Neurons (D19-1)
Copied to clipboard
| Challenge: | Recent studies have shown that a hybrid of self-attention networks (SANs) and recurrent neural networks (RNNs) outperforms both individual architectures, while not much is known about why the hybrid models work. |
| Approach: | They propose to use an advanced variant of self-attention networks (SANs) to enhance the strength of hybrid models by introducing a syntax-oriented inductive bias to perform tree-like composition. |
| Outcome: | The proposed model outperforms both individual models and a standard hybrid model on a machine translation task. |
BERT-enhanced Relational Sentence Ordering Network (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to improve coherence modeling for paragraphs have been developed. |
| Approach: | They propose a BERT-enhanced Relational Sentence Ordering Network to capture better dependency relationship among sentences and exploit it with a deep relational module. |
| Outcome: | The proposed model shows significant improvement over the state-of-the-art on six datasets. |
A Deep Neural Network Sentence Level Classification Method with Context Information (D18-1)
Copied to clipboard
| Challenge: | Existing methods that use context for sentence classification are difficult to scale . Usually, sentences are treated as separate instances for the task . however, in many situations the sentence that is the focus of classification appears in a context that can provide additional information. |
| Approach: | They propose a method that uses potentially large contexts to classify sentences . they use an LSTM, and short-span features to classize sentences based on a stacked CNN . |
| Outcome: | The proposed method consistently improves on two different datasets. |
Are BLEU and Meaning Representation in Opposition? (P18-1)
Copied to clipboard
| Challenge: | Empirical evaluation suggests that the better the translation quality, the worse the learned sentence representations serve in a wide range of classification and similarity tasks. |
| Approach: | They propose several variations of the attentive NMT architecture to bring this meeting point back . they propose to use a structured fixed-size representation of the input to produce static representations of input sentences. |
| Outcome: | The proposed architecture improves translation quality and performance in a range of tasks. |
Self-Attention with Structural Position Representations (D19-1)
Copied to clipboard
| Challenge: | Experimental results show that SANs can't encode positions of input words . SAN's are currently lacking in encoding positions of words based on position-unaware "bagof-words" theory . |
| Approach: | They propose to augment SANs with structural position representations to capture latent structure of input sentence. |
| Outcome: | The proposed approach consistently outperforms the sequential representations on translation tasks. |
A Richer-but-Smarter Shortest Dependency Path with Attentive Augmentation for Relation Extraction (N19-1)
Copied to clipboard
| Challenge: | Existing approaches to extract relationship between entities in sentences suffer from missing or redundant information. |
| Approach: | They propose a deep neural model that combines the advantages of the two approaches to extract the relationship between two entities in a sentence. |
| Outcome: | The proposed model outperforms baseline models on the SemEval-2010 dataset. |
Non-Autoregressive Sentence Ordering (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing sentence ordering approaches only leverage unilateral dependencies during decoding and cannot fully explore the semantic dependency between sentences. |
| Approach: | They propose a non-autoregressive ordering network that explores bilateral dependencies between sentences and predicts sentences for each position in parallel. |
| Outcome: | The proposed model outperforms existing autoregressive sentence ordering approaches and yields competitive performance compared with the state-of-the-arts. |
Phrase-level Self-Attention Networks for Universal Sentence Encoding (D18-1)
Copied to clipboard
| Challenge: | Phrase-level self-attention networks (PSAN) can capture context dependencies at the phrase level instead of the sentence level. |
| Approach: | They propose to perform self-attention across words inside a phrase to capture context dependencies at the phrase level and use the gated memory updating mechanism to refine each word’s representation hierarchically with longer-term context dependency captured in a larger phrase. |
| Outcome: | The proposed model can achieve state-of-the-art performance across a plethora of NLP tasks including binary and multi-class classification, natural language inference and sentence similarity. |
MODE-LSTM: A Parameter-efficient Recurrent Network with Multi-Scale for Sentence Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for sentence classification use linear convolution, which may not be sufficient to model the non-consecutive dependency of the phrase and may overfit the sequential information. |
| Approach: | They propose a model that extracts multi-scale n-gram features for understanding the semantic meaning of sentences by some key-phrases located at different positions. |
| Outcome: | The proposed model outperforms existing models on eight benchmark datasets and is competitive against state-of-the-art models. |