Papers with representations
SmBoP: Semi-autoregressive Bottom-up Semantic Parsing (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing semantic parsers decode syntax using a top-down depth-first traversal. |
| Approach: | They propose a semi-autoregressive bottom-up parser that constructs at decoding step t the top-K sub-trees of height t. |
| Outcome: | The proposed method achieves 2.2x speed-up in decoding time and 5x speed up in training time on a zero-shot semantic parsing benchmark. |
Deep Dirichlet Multinomial Regression (N18-1)
Copied to clipboard
| Challenge: | supervised topic models can incorporate arbitrary document-level features to inform topic priors, but their ability to model corpora is limited by the representation and selection of these features. |
| Approach: | They propose a generative topic model that simultaneously learns document feature representations and topics. |
| Outcome: | The proposed model outperforms DMR and LDA on three datasets and human subjects judge it more representative of associated document features. |
Data Selection for Fine-tuning Large Language Models Using Transferred Shapley Values (2023.acl-srw)
Copied to clipboard
| Challenge: | Large language models (LMs) have been shown to be highly effective for identifying harmful training instances, but dataset size and model complexity constraints limit the ability to apply Shapley-based data valuation to fine-tuning large pre-trained language models. |
| Approach: | They propose an algorithm that aggregates Shapley values from subsets for valuation of entire training set and a value transfer method that leverages value information extracted from a simple classifier trained using representations from the target language model. |
| Outcome: | The proposed method outperforms existing methods on benchmark datasets and can filter fine-tuning data to increase language model performance compared to training with the full fine-uning dataset. |
The Devil is in the Distributions: Explicit Modeling of Scene Content is Key in Zero-Shot Video Captioning (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for zero-shot video captioning focus on one key aspect of the scene and ignore the rest of the visual input. |
| Approach: | They propose a novel textual prompting strategy for zero-shot video captioning that uses a category-aware retrieval mechanism to promote prompt diversity while ensuring visual relevance. |
| Outcome: | The proposed method outperforms existing methods on in-domain and cross-domain settings. |
Convergent Demographic Utility Hierarchies: Geometry of Intersectional Values in LLMs (2026.acl-srw)
Copied to clipboard
| Challenge: | Recent work shows that LLMs develop internally coherent utility functions that emerge with scale. |
| Approach: | They elicit pairwise preferences across 15 intersectional demographic groups . they fit Thurstonian utility functions to the preference matrices and find a compensatory hierarchy . |
| Outcome: | elicited preferences show that they encode demographic hierarchies across 15 demographic groups . gender, race, and combinations overestimate the most extreme intersectional gap by 26- 40% . |
Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent work analyzes, quantifies, and mitigates language model biases such as gender, race or religion-related stereotypes in static word embeddings and contextual representations. |
| Approach: | They explain the complexity of gender and language around it and examine how current representations perpetuate harms associated with binary gender. |
| Outcome: | The proposed model and dataset biases perpetuate harms associated with the treatment of gender as binary in English language technologies. |
Learned Incremental Representations for Parsing (2022.acl-long)
Copied to clipboard
| Challenge: | a new syntactic representation that commits to syntakic choices is proposed for humans . we use a system that uses only incremental processing of a prefix to predict the word in a sentence . |
| Approach: | They propose a syntactic representation that commits to syntakic choices incrementally . they say the system can achieve 93.72 F1 on the Penn Treebank with as few as 5 bits per word . |
| Outcome: | The proposed representation achieves 93.72 F1 on the Penn Treebank with as few as 5 bits per word . the analysis of the representations shows they have interpretable features and deferred resolution of syntactic ambiguities. |
Fine-tuning Encoders for Improved Monolingual and Zero-shot Polylingual Neural Topic Modeling (2021.naacl-main)
Copied to clipboard
| Challenge: | Topic models can augment or replace bag-of-words inputs with pre-trained transformer-based word prediction models. |
| Approach: | They propose several methods for fine-tuning encoders to improve both monolingual and zero-shot polylingual topic modeling. |
| Outcome: | The proposed methods improve both monolingual and zero-shot polylingual topic modeling. |
Query-focused Referentiability Learning for Zero-shot Retrieval (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing dense representations have shown limitations in zero-shot scenarios . however, passage representations fail to align with their gold queries . |
| Approach: | They propose a query-focused concept of 'referentiable' which ensures passage representations are referenced by their gold queries. |
| Outcome: | The proposed model outperforms existing models on the BEIR benchmark. |
Probing for idiomaticity in vector space models (2021.eacl-main)
Copied to clipboard
| Challenge: | Contextualised word representation models are used to represent idiomaticity in language. |
| Approach: | They propose probing measures to assess if some of the expected linguistic properties of noun compounds are readily available in some standard and widely used representations. |
| Outcome: | The proposed models show that idiomaticity is not yet accurately represented by contextualised models. |
DiffCSE: Difference-based Contrastive Learning for Sentence Embeddings (2022.naacl-main)
Copied to clipboard
Yung-Sung Chuang, Rumen Dangovski, Hongyin Luo, Yang Zhang, Shiyu Chang, Marin Soljacic, Shang-Wen Li, Scott Yih, Yoon Kim, James Glass
| Challenge: | Recent work shows that finetuning pretrained models with contrastive learning makes it possible to learn good sentence embeddings without labeled data. |
| Approach: | They propose an unsupervised contrastive learning framework for learning sentence embeddings . they use a masked language model to mask out the edited sentence . |
| Outcome: | The proposed framework outperforms SimCSE on semantic textual similarity tasks by 2.3 absolute points. |
Learning Disentangled Representations of Texts with Application to Biomedical Abstracts (D18-1)
Copied to clipboard
| Challenge: | a method for learning disentangled representations of texts that encode distinct and complementary aspects is proposed . a classic problem in distributed representation learning is that it is difficult to determine what information individual dimensions encode. |
| Approach: | They propose a method for learning disentangled representations of texts that encode distinct and complementary aspects by a adversarial objective based on the (dis)similarity between triplets of documents with respect to specific aspects. |
| Outcome: | The proposed method can be used to perform aspect-specific retrieval on biomedical abstracts. |
How to Make Large Language Models Generate 100% Valid Molecules? (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) can learn to perform a wide range of tasks, but generating valid molecules using representations like SMILES is challenging in few-shot settings. |
| Approach: | They propose a language framework that converts invalid SMILES to SELFIES and LLMs as post-hoc correctors to ensure that the molecules generated by LLM are 100% valid. |
| Outcome: | The proposed model performs worse with SELFIES than with SMILES and improves on other metrics. |