Co-Stack Residual Affinity Networks with Multi-level Attention Refinement for Matching Text Sequences

Challenge:	a long standing problem in NLP research is learning a matching function between two text sequences . a deep architecture for this task is proposed by a team of researchers .
Approach:	They propose a new deep matching model using stacked recurrent encoders to learn affinity weights . they conduct extensive experiments on six well-studied text sequence matching datasets a plethora of applications are possible .
Outcome:	The proposed model improves performance on six well-studied text sequence matching datasets.

Similar Papers

Overview

Challenge

Approach

Outcome

Recurrent Attention Networks for Long-text Modeling (2023.findings-acl)

Copied to clipboard

Xianming Li, Zongxi Li, Xiaotian Luo, Haoran Xie, Xing Lee, Yingbin Zhao, Fu Lee Wang, Qing Li

Information Extraction & Text Mining

Challenge:	Existing approaches to encoding long documents using self-attention have been limited by quadratic computational complexities and limited application in long text processing.
Approach:	They propose a long-document encoding model that allows the recurrent operation of self-attention.
Outcome:	The proposed model extracts global semantics in token-level and document-level representations, making it inherently compatible with both sequential and sequential tasks.

Parallel Attention Network with Sequence Matching for Video Grounding (2021.findings-acl)

Copied to clipboard

Hao Zhang, Aixin Sun, Wei Jing, Liangli Zhen, Joey Tianyi Zhou, Siow Mong Rick Goh

Multimodality

Challenge:	Existing approaches to video grounding are sensitive to quality of proposals and inefficient because all proposal-query pairs are compared.
Approach:	They propose a Parallel Attention Network with Sequence matching to capture selfmodal contexts and cross-modal attentive information between video and text.
Outcome:	The proposed approach is superior to state-of-the-art methods on three datasets.

Phrase-level Self-Attention Networks for Universal Sentence Encoding (D18-1)

Copied to clipboard

Wei Wu, Houfeng Wang, Tianyu Liu, Shuming Ma

Semantic Text Processing

Challenge:	Phrase-level self-attention networks (PSAN) can capture context dependencies at the phrase level instead of the sentence level.
Approach:	They propose to perform self-attention across words inside a phrase to capture context dependencies at the phrase level and use the gated memory updating mechanism to refine each word’s representation hierarchically with longer-term context dependency captured in a larger phrase.
Outcome:	The proposed model can achieve state-of-the-art performance across a plethora of NLP tasks including binary and multi-class classification, natural language inference and sentence similarity.

Context-Aware Interaction Network for Question Matching (2021.emnlp-main)

Copied to clipboard

Zhe Hu, Zuohui Fu, Yu Yin, Gerard de Melo

Challenge:	Existing models focus on word-level local matching and neglect the importance of contextual information.
Approach:	They propose a context-aware interaction network to properly align two sequences and infer their semantic relationship by using gate fusion layers.
Outcome:	The proposed model can accurately align two sequences and infer their semantic relationship on two question matching datasets.

Original Semantics-Oriented Attention and Deep Fusion Network for Sentence Matching (D19-1)

Copied to clipboard

Mingtong Liu, Yujie Zhang, Jinan Xu, Yufeng Chen

Challenge:	Sentence matching is a key issue in natural language inference and paraphrase identification.
Approach:	They propose a semantics-oriented attention and deep fusion network (OSOA-DFN) that is oriented to the original semantic representation of another sentence and propagates attention information at each matching layer.
Outcome:	The proposed model can model sentence matching more precisely on three sentence matching benchmark datasets.

Multi-Granularity Hierarchical Attention Fusion Networks for Reading Comprehension and Question Answering (P18-1)

Copied to clipboard

Wei Wang, Ming Yan, Chen Wu

Reasoning

Challenge:	Existing approaches to read comprehension style question answering are limited by the volume of annotated datasets.
Approach:	They propose a hierarchical attention network for reading comprehension style question answering . they first encode the question and paragraph with fine-grained language embeddings . then propose fusion approach to fuse information from both global and attended representations based on the hierarchic attention network .
Outcome:	The proposed method achieves state-of-the-art on the SQuAD and TriviaQA Wiki leaderboards and two adversarial SQu AD datasets.

A Deep Neural Information Fusion Architecture for Textual Network Embeddings (D19-1)

Copied to clipboard

Zenan Xu, Qinliang Su, Xiaojun Quan, Weijia Zhang

Representation Learning

Challenge:	Textual network embeddings aim to learn a low-dimensional representation for every node in the network while seeking to retain the original network information.
Approach:	They propose a deep neural architecture to fuse the two kinds of informations into one representation.
Outcome:	The proposed model outperforms the comparing methods on all three datasets.

Self-Attentive Residual Decoder for Neural Machine Translation (N18-1)

Copied to clipboard

Lesly Miculicich Werlen, Nikolaos Pappas, Dhananjay Ram, Andrei Popescu-Belis

Language Models

Challenge:	Neural sequence-to-sequence networks with attention have been used for machine translation . however, the target-side context is limited and the model lacks the ability to capture non-syntactic dependencies among words.
Approach:	They propose a sequence-to-sequence network with attention that captures contextual information at each time-step prediction through an attention mechanism.
Outcome:	The proposed model outperforms a neural MT baseline and memory and self-attention network on three language pairs.

A Lightweight Recurrent Network for Sequence Modeling (P19-1)

Copied to clipboard

Biao Zhang, Rico Sennrich

Green & Sustainable NLP

Challenge:	Recent studies show that recurrent networks suffer from severe computational inefficiency due to weak parallelization.
Approach:	They propose a lightweight recurrent network (LRN) that uses input and forget gates to handle long-range dependencies and gradient vanishing and explosion.
Outcome:	The proposed recurrent network yields the best running efficiency on six NLP tasks.

Simple and Effective Text Matching with Richer Alignment Features (P19-1)

Copied to clipboard

Runqi Yang, Jianhai Zhang, Xing Gao, Feng Ji, Haiqing Chen

Challenge:	Existing models only use a single inter-sequence alignment layer to make full use of this process.
Approach:	They propose to keep three key features available for inter-sequence alignment . they conduct experiments on four well-studied benchmark datasets .
Outcome:	The proposed model is able to perform on four well-studied datasets with fewer parameters and the inference speed is at least 6 times faster than similar models.