| Challenge: | Currently, little attention is paid to how to aggregate text sequences into fixed-size vectors. |
| Approach: | They propose an aggregation mechanism to obtain a fixed-size encoding with a dynamic routing policy. |
| Outcome: | The proposed method outperforms other aggregating methods on five text classification tasks. |
Similar Papers
Investigating Capsule Networks with Dynamic Routing for Text Classification (D18-1)
Copied to clipboard
| Challenge: | Earlier efforts in text modeling have achieved limited success on word meanings . convolutional neural networks (CNNs) are used to model higher level concepts and facts in texts . |
| Approach: | They propose three strategies to stabilize dynamic routing process to alleviate disturbance of noise capsules. |
| Outcome: | The proposed methods achieve state-of-the-art on 4 out of 6 datasets . they show that capsule networks exhibit significant improvement over baseline methods . |
Towards Linear Time Neural Machine Translation with Capsule Networks (D19-1)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) is an endto-end learning approach to machine translation. |
| Approach: | They propose a capsule network with dynamic routing for linear time Neural Machine Translation . they map the source sentence into a matrix with pre-determined size and apply a deep LSTM network to decode the target sequence from the source representation. |
| Outcome: | The proposed network achieves comparable results with the Transformer system on English-German and English-French tasks. |
Lightweight, Dynamic Graph Convolutional Networks for AMR-to-Text Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | AMR-to-text generation is used to transduce Abstract Meaning Representation structures (AMRs) Graph Convolution Networks (GCNs) are not able to capture non-local information and follow a local (first-order) information aggregation scheme. |
| Approach: | They propose a dynamic fusion mechanism that captures richer non-local interactions . they propose weight tied convolutions and group graph convolution to reduce memory usage . |
| Outcome: | The proposed model outperforms state-of-the-art models on two benchmark datasets with significantly fewer parameters while maintaining the model capacity. |
Aggretriever: A Simple Approach to Aggregate Textual Representations for Robust Dense Passage Retrieval (2023.tacl-1)
Copied to clipboard
| Challenge: | Pre-trained language models have been successful in knowledge-intensive tasks, but recent research calls into question the robustness of these singlevector models. |
| Approach: | They propose to exploit knowledge in a pre-trained language model for dense passage retrieval by aggregating contextualized token embeddings into a dense vector. |
| Outcome: | The proposed model significantly improves the effectiveness of dense retrieval models on in-domain and zero-shot evaluations without introducing substantial training overhead. |
Vector of Locally Aggregated Embeddings for Text Representation (N19-1)
Copied to clipboard
| Challenge: | Existing models for text classification use word embeddings, weighted averaging, and deepening networks. |
| Approach: | They propose a vector-based locally averaging model that encodes each input text by effectively identifying and integrating its semantically-relevant parts. |
| Outcome: | The proposed model outperforms RNNs and CNNs in text classification while taking only a fraction of their training time. |
Vector of Locally-Aggregated Word Embeddings (VLAWE): A Novel Document-level Representation (N19-1)
Copied to clipboard
| Challenge: | a novel word embedding representation for text documents is proposed . the method is based on the Vector of Locally-Aggregated Descriptors used for image representation . |
| Approach: | They propose a novel representation for text documents based on aggregating word embedding vectors into document embeddables. |
| Outcome: | The proposed representation improves on the Movie Review data set and is 10% better than the state-of-the-art representation. |
Hierarchical Bracketing Encodings Work for Dependency Graphs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Sequence labeling (SL) is a simple yet effective paradigm for a wide range of natural language problems. |
| Approach: | They propose a new bracketing approach for dependency graph parsing that encodes graphs as sequences and n tagging actions. |
| Outcome: | The proposed approach significantly reduces label space while preserving structural information. |
OptSLA: an Optimization-Based Approach for Sequential Label Aggregation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to annotate data are based on binary and multi-choice problems, but little work has been done on complex tasks such as sequence labeling with imbalanced classes. |
| Approach: | They propose an optimization-based Sequential Label Aggregation method that integrates worker reliabilities and advanced deep learning techniques to conquer the challenge. |
| Outcome: | The proposed method outperforms state-of-the-art methods and is easier to interpret. |
Bringing Emerging Architectures to Sequence Labeling in NLP (2026.eacl-long)
Copied to clipboard
| Challenge: | Pretrained Transformer encoders are the dominant approach to sequence labeling . however, few have been applied to sequence labels on flat or simplified tasks . |
| Approach: | They propose to use pretrained Transformer encoders to model relations across words . they find that the architectures adapt well across tagging tasks that vary in complexity . |
| Outcome: | The proposed architectures perform well across tagging tasks across languages and datasets. |
Semi-Structured Object Sequence Encoders (2023.findings-emnlp)
Copied to clipboard
Rudra Murthy, Riyaz Bhat, Chulaka Gunasekara, Siva Patel, Hui Wan, Tejas Dhamecha, Danish Contractor, Marina Danilevsky
| Challenge: | Semi-structured object sequences are often represented as a sequence of key-value pairs over time . authors propose a two-part approach that takes each key independently and encodes a representation of its values over time. |
| Approach: | They propose a two-part approach that first considers each key independently and encodes a representation of its values over time. |
| Outcome: | The proposed approach outperforms existing methods on multiple prediction tasks using real-world data. |