Original Semantics-Oriented Attention and Deep Fusion Network for Sentence Matching (D19-1)
Copied to clipboard
| Challenge: | Sentence matching is a key issue in natural language inference and paraphrase identification. |
| Approach: | They propose a semantics-oriented attention and deep fusion network (OSOA-DFN) that is oriented to the original semantic representation of another sentence and propagates attention information at each matching layer. |
| Outcome: | The proposed model can model sentence matching more precisely on three sentence matching benchmark datasets. |
Similar Papers
DABERT: Dual Attention Enhanced BERT for Semantic Matching (2022.coling-1)
Copied to clipboard
| Challenge: | Existing models for semantic sentence matching lack the ability to capture subtle differences. |
| Approach: | They propose to use a Transformer-based pre-trained language model to capture fine-grained differences in sentence pairs by introducing a dual attention module and a fusion module to learn the aggregation of difference and affinity features. |
| Outcome: | The proposed method is able to capture fine-grained differences in sentence pairs. |
Context-Aware Interaction Network for Question Matching (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing models focus on word-level local matching and neglect the importance of contextual information. |
| Approach: | They propose a context-aware interaction network to properly align two sequences and infer their semantic relationship by using gate fusion layers. |
| Outcome: | The proposed model can accurately align two sequences and infer their semantic relationship on two question matching datasets. |
Cross-sentence Pre-trained Model for Interactive QA matching (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for semantic matching do not examine each sentence individually, but consider syntactic context inside a sentence. |
| Approach: | They propose a semantic matching model that takes a cross-sentence context-aware architecture and incorporates a quantity of context information jump to facilitate attention weight formulation. |
| Outcome: | The proposed model outperforms state-of-the-art models on the Yahoo! community question dataset and the TREC library. |
Improving Semantic Matching through Dependency-Enhanced Pre-trained Model with Adaptive Fusion (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on dependency prior structure integration into pre-trained models is still unclear. |
| Approach: | They propose a dependency-based fusion attention paradigm which explicitly introduces dependency prior structure into pre-trained models and adaptively fuses it with semantic information. |
| Outcome: | The proposed model achieves state-of-the-art or competitive performance on 10 public datasets, demonstrating the benefits of adaptively fusing dependency structure in semantic matching task. |
Deep Attentive Sentence Ordering Network (D18-1)
Copied to clipboard
| Challenge: | Existing methods for sentence ordering tasks rely on linguistic knowledge and are domain specific. |
| Approach: | They propose a deep attentive sentence ordering network which integrates self-attention mechanism with LSTMs in the encoding of input sentences. |
| Outcome: | The proposed model outperforms the state-of-the-art models on Sentence Ordering and Order Discrimination tasks and is shown to be highly efficient. |
Phrase-level Self-Attention Networks for Universal Sentence Encoding (D18-1)
Copied to clipboard
| Challenge: | Phrase-level self-attention networks (PSAN) can capture context dependencies at the phrase level instead of the sentence level. |
| Approach: | They propose to perform self-attention across words inside a phrase to capture context dependencies at the phrase level and use the gated memory updating mechanism to refine each word’s representation hierarchically with longer-term context dependency captured in a larger phrase. |
| Outcome: | The proposed model can achieve state-of-the-art performance across a plethora of NLP tasks including binary and multi-class classification, natural language inference and sentence similarity. |
Structured Alignment Networks for Matching Sentences (D18-1)
Copied to clipboard
| Challenge: | Many tasks in natural language processing involve comparing two sentences to compute some notion of relevance, entailment, or similarity. |
| Approach: | They propose a model of structured alignments between sentences to compare two sentences by matching their latent structures. |
| Outcome: | The proposed model is differentiable and trained only on the matching objective. |
Sentence-Level Evidence Embedding for Claim Verification with Hierarchical Attention Networks (P19-1)
Copied to clipboard
| Challenge: | Claim verification is cumbersome and inefficient for human fact-checkers to find consistent pieces of evidence. |
| Approach: | They propose an end-to-end hierarchical attention network that learns to represent coherent evidence and their semantic relatedness with the claim. |
| Outcome: | The proposed model outperforms state-of-the-art models on three datasets . it is based on a coherence-based attention layer and entailment-based one . |
Convolutional Interaction Network for Natural Language Inference (D18-1)
Copied to clipboard
| Challenge: | Attention-based neural models have achieved great success in natural language inference (NLI). |
| Approach: | They propose a general model to capture the interaction between two sentences, which can be an alternative to the attention mechanism for NLI. |
| Outcome: | The proposed model can capture complex interactions on three large datasets. |
MATCH: Modulating Attention via In-Context Retrieval for Long-Context Transformers (2026.acl-long)
Copied to clipboard
Linrui Ma, Chun Hei Lo, Xinyu Wang, Peng Lu, Xihao Yuan, Hanting Chen, Kai Han, Xinghao Chen, Chengjun Zhan, Hanlin xu, Yichun Yin, Lifeng Shang, Feng Wen, Boxing Chen, Yufei Cui
| Challenge: | Existing approaches to improve efficiency often enforce rigid structural constraints such as local attention windows. |
| Approach: | They propose a framework that augments sparse-attention mechanisms with dynamically integrated in-context information through an efficient retrieval system. |
| Outcome: | Empirical results show that MATCH significantly improves the performance of sparse-attention models on synthetic and real-world natural-language tasks. |