Papers with pooling
On the Effectiveness of the Pooling Methods for Biomedical Relation Extraction with Deep Learning (D19-62)
Copied to clipboard
| Challenge: | Existing models for relation extraction use different pooling mechanisms to perform pooling for RE. |
| Approach: | They conduct a comprehensive study to evaluate the effectiveness of different pooling mechanisms for deep learning in biomedical RE. |
| Outcome: | The proposed model outperforms the previous models on two biomedical datasets. |
Correlations between Word Vector Sets (D19-1)
Copied to clipboard
| Challenge: | Similarity measures based on word embeddings are easily competing with deep learning and expert-engineered systems on unsupervised semantic textual similarity tasks. |
| Approach: | They propose a new approach to measure word embeddings using pooling operations and correlation coefficients instead of pooling . they also propose centered kernel alignment as a natural generalisation of squared cosine similarity for sets of word vectors. |
| Outcome: | The proposed approach outperforms most recent methods while being much faster and trivial to implement. |
Distance-Free Modeling of Multi-Predicate Interactions in End-to-End Japanese Predicate-Argument Structure Analysis (C18-1)
Copied to clipboard
| Challenge: | Existing models for analyzing PASs in Japanese are lacking in identifying elliptical arguments. |
| Approach: | They propose to extend the input and last layers of a bidirectional recurrent neural network model to capture the potential interactions among multiple PASs. |
| Outcome: | The proposed models improve prediction accuracy on a benchmark corpus and achieve state-of-the-art on standardized corpus. |
Formal Semantic Controls over Language Models (2024.lrec-tutorials)
Copied to clipboard
| Challenge: | Text embeddings provide a concise representation of the semantics of sentences and larger spans of text, rather than individual words, capturing a wide range of linguistic features. |
| Approach: | They propose to shorten the gap between latent semantics and formal symbolics by comparing distributional models to symbolic models grounded on formal linguistics and well-defined mathematical properties. |
| Outcome: | This paper examines the analysis and control of text representations, covering methods from pooling to LLM-based. |
Delexicalized Paraphrase Generation (2020.coling-industry)
Copied to clipboard
| Challenge: | Using convolutional neural networks, we generate delexicalized sentences . 1.29% accuracy is achieved with the generated paraphrases . |
| Approach: | They propose a neural paraphrasing model that generates delexicalized sentences . they use convolutional neural networks to pool on slot values and use pointers to locate them . |
| Outcome: | The proposed model generates delexicalized sentences with high quality . it can be used for intent classification and named entity recognition tasks . |
Improving Visual-Semantic Embedding with Adaptive Pooling and Optimization Objective (2023.eacl-main)
Copied to clipboard
Zijian Zhang, Chang Shu, Ya Xiao, Yuan Shen, Di Zhu, Youxin Chen, Jing Xiao, Jey Han Lau, Qian Zhang, Zheng Lu
| Challenge: | Recent VSE models combine simple pooling methods with hard triplet loss to improve performance. |
| Approach: | They propose an adaptive pooling strategy that allows the model to learn how to aggregate features through a combination of simple pooling methods. |
| Outcome: | The proposed strategy outperforms current state-of-the-art systems on image-to-text and text-toimage retrieval. |
Subword Pooling Makes a Difference (2021.eacl-main)
Copied to clipboard
| Challenge: | Contextual word-representations use subword tokenization to handle large vocabularies and unknown words. |
| Approach: | They propose to use the first subword for morphological probing, POS tagging and NER to pool multiple subwords that correspond to a single word in contextual language models. |
| Outcome: | The proposed model outperforms two multilingual models on morphological probing, POS tagging and NER tasks in 9 languages. |
CASE – Condition-Aware Sentence Embeddings for Conditional Semantic Textual Similarity Measurement (2026.eacl-long)
Copied to clipboard
| Challenge: | Recent approaches use semantic similarity to improve the quality of sentence embeddings, but it is difficult to measure the similarity between sentences. |
| Approach: | They propose a condition-aware sentence embedding method that uses an LLM encoder to create an embeddable sentence under a given condition. |
| Outcome: | The proposed method improves the performance of LLM-based embeddings and the isotropy of the embeddable space despite requiring a small number of dimensions. |
Attentive Pooling with Learnable Norms for Text Representation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing pooling methods that use fixed pooling norms may not be optimal for learning text representations in different tasks. |
| Approach: | They propose to learn pooling norms in an end-to-end manner to automatically find the optimal ones for text representation in different tasks. |
| Outcome: | The proposed approach improves on four benchmark datasets on a neural NLP model. |
Hyperbolic Capsule Networks for Multi-Label Classification (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for classification of labels are limited by feature aggregation and encoding. |
| Approach: | They propose to use hyperbolic capsule networks to capture fine-grained label information . they also propose a new routing method to adaptively adjust capsule number during routing . |
| Outcome: | The proposed method significantly improves the performance of multi-label classification on tail labels. |
Attention for Implicit Discourse Relation Recognition (L18-1)
Copied to clipboard
| Challenge: | Existing approaches to implicit discourse relation recognition reach F1 scores of 9.95% to 37.67% . a neural network exploits the strong correlation between pairs of words that implicitly signal a discourse relation. |
| Approach: | They propose a neural network which exploits strong correlation between pairs of words . they use an encoder-decoder model with attention to detect a latent discourse relation . |
| Outcome: | The proposed model outperforms state-of-the-art models on fine-grained classification and fine-granular classification while computing parameters without pooling and fully connected layers. |
SciFact-Open: Towards open-domain scientific claim verification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Current scientific claim verification systems can achieve very strong performance on limited contexts, in some cases approaching human agreement. |
| Approach: | They propose to pool and annotate top predictions from four state-of-the-art scientific claim verification models to evaluate their performance against large corpora. |
| Outcome: | The proposed system performs well on a corpus of 500K scientific abstracts. |
Why and when should you pool? Analyzing Pooling in Recurrent Architectures (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Various pooling techniques have been shown to improve performance of RNNs on text classification tasks. |
| Approach: | They propose a pooling-based variant that captures interactions among predictive tokens in a sentence. |
| Outcome: | The proposed pooling architecture outperforms non-pooling models on sequence classification tasks. |
Interpreting Pretrained Contextualized Representations via Reductions to Static Embeddings (2020.acl-main)
Copied to clipboard
| Challenge: | Contextualized representations have become the default for downstream NLP applications. |
| Approach: | They propose a method for converting from contextualized representations to static lookup-table embeddings and apply it to 5 popular pretrained models and 9 sets of pretrained weights. |
| Outcome: | The proposed methods show that pooling over many contexts significantly improves representational quality under intrinsic evaluation. |
Multi-Vector Attention Models for Deep Re-ranking (2021.emnlp-main)
Copied to clipboard
| Challenge: | Document retrieval systems often use two styles of neural network models . dual encoder models are used for retrieval and deep re-ranking, while cross-attention models are typically used for shallow reranking. |
| Approach: | They propose a dual encoder and cross-attention neural network architectures that combine query and document representations to optimize retrieval accuracy. |
| Outcome: | The proposed architecture trades off retrieval accuracy with joint computation and offline document storage cost. |
Label and Explanation Variation in LLM-Based Annotation: a Case Study in Natural Language Inference (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown considerable promise for annotation purposes, but questions remain about their ability to capture human label variation (HLV) label variation is genuine disagreement between annotators observed across NLP tasks. |
| Approach: | They investigate how label and explanation variation manifests within and across LLMs with respect to the Natural Language Inference task. |
| Outcome: | The proposed models generate label distributions similar to humans but exhibit distinct, idiosyncratic judgments and disagreement patterns. |
Crowdsourcing Latin American Spanish for Low-Resource Text-to-Speech (2020.lrec-1)
Copied to clipboard
Adriana Guevara-Rukoz, Isin Demirsahin, Fei He, Shan-Hui Cathy Chu, Supheakmungkol Sarin, Knot Pipatsrisawat, Alexander Gutkin, Alena Butryna, Oddur Kjartansson
| Challenge: | Using crowd-sourced datasets, we build a text-to-speech voice for a new dialect in a language with existing resources. |
| Approach: | They propose a multidialectal corpus approach for building a text-to-speech voice for a new dialect in a language with existing resources using crowd-sourcing. |
| Outcome: | The proposed model outperforms baseline models in a “zero-resource” dialect scenario while holding out target dialect recordings from the training data. |