Papers with convolution
LV-BERT: Exploiting Layer Variety for BERT (2021.findings-acl)
Copied to clipboard
| Challenge: | Modern pre-trained language models are mostly built upon stereotyped development sets . LV-BERT model obtained by our method outperforms BERT on various downstream tasks . |
| Approach: | They propose to exploit layer variety from the layer type set and the layer order to improve pre-trained models. |
| Outcome: | The proposed model outperforms BERT and its variants on various downstream tasks. |
TwiRGCN: Temporally Weighted Graph Convolution for Question Answering over Temporal Knowledge Graphs (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent years have witnessed interest in Temporal Question Answering over Knowledge Graphs (TKGQA) but these methods are highly engineered and do not automatically discover relevant parts of the KG during multi-hop reasoning. |
| Approach: | They propose a scheme to modulate the messages passed through a KG edge during convolution based on the relevance of its associated period to the question. |
| Outcome: | The proposed system outperforms state-of-the-art models on a recent challenging dataset for multi-hop complex temporal QA called TimeQuestions. |
GL-GAN: Perceiving and Integrating Global and Local Styles for Handwritten Text Generation with Mamba (2025.coling-main)
Copied to clipboard
| Challenge: | Existing models lack the ability to perceive and integrate handwriting styles, which affects the realism of the synthesized samples. |
| Approach: | They propose a Hybrid Style Encoder that captures global and local styles and integrates them into a Dynamic Feature Enhancement Module (DFEM). |
| Outcome: | The proposed model outperforms state-of-the-art models on two widely used handwriting datasets and can provide training data for handwritten text recognition and signature verification. |
mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understanding (2024.findings-emnlp)
Copied to clipboard
Anwen Hu, Haiyang Xu, Jiabo Ye, Ming Yan, Liang Zhang, Bo Zhang, Ji Zhang, Qin Jin, Fei Huang, Jingren Zhou
| Challenge: | Existing Multimodal Large Language Models lack general structure understanding abilities for text-rich document images. |
| Approach: | They propose to use unified structure learning to boost the performance of MLLMs by encoding structure information into text-rich images. |
| Outcome: | The proposed model achieves state-of-the-art on 10 visual document understanding benchmarks. |
ConvTextTM: An Explainable Convolutional Tsetlin Machine Framework for Text Classification (2022.lrec-1)
Copied to clipboard
| Challenge: | Recent advances in natural language processing (NLP) have reshaped the industry . complexity of such models makes them a “black box” and can cause ethical concerns . |
| Approach: | They propose a convolutional TM architecture that breaks down text into a sequence of fragments . they propose to use a tokenization scheme to bind the tokens to the text fragments. |
| Outcome: | The proposed architecture improves on a set of text fragments and eliminates the need for a corpus-specific vocabulary. |
SMR: State Memory Replay for Long Sequence Modeling (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing state space models (SSMs) address non-uniform sampling, but their recursive structures impede efficient SSM computation via convolution. |
| Approach: | They propose a plug-and-play mechanism to solve the Non-Stable State problem by adjusting input sequences with early memories. |
| Outcome: | The proposed method overcomes the non-uniform sample processing problem . it can achieve Sampling Step Adaptation (SSA) by adjusting input sequences with early memories. |
DAPE V2: Process Attention Score as Feature Map for Length Extrapolation (2025.acl-long)
Copied to clipboard
Chuanyang Zheng, Yihang Gao, Han Shi, Jing Xiong, Jiankai Sun, Jingyao Li, Minbin Huang, Xiaozhe Ren, Michael Ng, Xin Jiang, Zhenguo Li, Yu Li
| Challenge: | Extensive experiments demonstrate that treating attention as a feature map and applying convolution as . a processing method significantly enhances Transformer performance. |
| Approach: | They propose to use the convolution operator to mimic the processing methods in computer vision to treat attention as a feature map and apply it to neighboring attention scores across different heads. |
| Outcome: | The proposed model can be adapted to various attention-related models and achieves high performance. |
Aspect-Level Sentiment Analysis Via Convolution over Dependency Tree (D19-1)
Copied to clipboard
| Challenge: | Existing methods to identify sentiment polarity of opinion words are cumbersome due to the amount of opinionated material on the internet. |
| Approach: | They propose a method to identify sentiment polarity of opinion words on a specific aspect of a sentence using neural networks. |
| Outcome: | The proposed method is the state-of-the-art in aspect-based sentiment classification. |
Learning distributed sentence vectors with bi-directional 3D convolutions (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods that render words or characters into images separately, but instead use text's visual features as input, we use 3-dimensional convolutions to learn distributed sentence representation. |
| Approach: | They propose to use text's visual features as input to learn distributed sentence representation using 3-dimensional sentence tensors and multiple 3-dimensional convolutions with different lengths are applied to the sentence . |
| Outcome: | The proposed model performs well on several downstream natural language processing tasks. |
DialogConv: A Lightweight Fully Convolutional Network for Multi-view Response Selection (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing retrieval-based dialogue systems suffer from slow inference or huge number of parameters. |
| Approach: | They propose a lightweight fully convolutional architecture for response selection using convolution. |
| Outcome: | The proposed architecture extracts matching features of context and response from 3D views. |