Varying Sentence Representations via Condition-Specified Routers (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing sentences cannot account for different aspects of semantic similarity between two sentences. |
| Approach: | They propose a transformer-style framework that generates conditioned sentences . they propose 'conditional' STS, which measures similarity between two sentences based on condition sentences - a task that requires a sentence embedding model capable of generating distinct representations for the same sentence under different conditions. |
| Outcome: | The proposed framework is superior to existing models on two condition sentences . it can generate conditioned sentences while maintaining model parameters and computational efficiency . |
Similar Papers
A Bilingual Generative Transformer for Semantic Sentence Embedding (2020.emnlp-main)
Copied to clipboard
| Challenge: | Semantic sentence embedding models encode natural language sentences into vectors, such that closeness in embeddable space indicates closeness of semantics between the sentences. |
| Approach: | They propose a deep latent variable model that attempts to perform source separation on parallel sentences, isolating what they have in common in a latent semantic vector, and explaining what is left over with language-specific latent vectors. |
| Outcome: | The proposed model outperforms the state-of-the-art on a standard suite of unsupervised semantic similarity evaluations. |
Hyper-CL: Conditioning Sentence Representations with Hypernetworks (2024.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to sentence embeddings do not capture fine-grained semantics of sentences. |
| Approach: | They propose a method that integrates hypernetworks with contrastive learning to generate conditioned sentence representations. |
| Outcome: | The proposed method narrows the performance gap with the bi-encoder architecture while maintaining the time efficiency characteristic of the tri-encoding approach. |
CASE – Condition-Aware Sentence Embeddings for Conditional Semantic Textual Similarity Measurement (2026.eacl-long)
Copied to clipboard
| Challenge: | Recent approaches use semantic similarity to improve the quality of sentence embeddings, but it is difficult to measure the similarity between sentences. |
| Approach: | They propose a condition-aware sentence embedding method that uses an LLM encoder to create an embeddable sentence under a given condition. |
| Outcome: | The proposed method improves the performance of LLM-based embeddings and the isotropy of the embeddable space despite requiring a small number of dimensions. |
Contextualized Embeddings based Transformer Encoder for Sentence Similarity Modeling in Answer Selection Task (2020.lrec-1)
Copied to clipboard
| Challenge: | Word embeddings that consider context have attracted great attention for natural language processing tasks in recent years. |
| Approach: | They propose two different approaches to integrate contextualized word embeddings with transformer encoders for sentence similarity modeling. |
| Outcome: | The proposed model outperforms the feature-based approach on six datasets. |
Using Semantic Similarity as Reward for Reinforcement Learning in Sentence Generation (P19-2)
Copied to clipboard
| Challenge: | Existing models for sentence generation use cross-entropy loss as the loss function . however, cross-etropy is unable to evaluate sentences as a whole and lacks flexibility . et al., 2018: a novel approach to improve sentence generation models . |
| Approach: | They propose a method to train a model using estimated semantic similarity between output and reference sentences to alleviate cross-entropy loss problems. |
| Outcome: | The proposed model improves the BLEU scores from the baseline LSTM NMT model. |
INSET: Sentence Infilling with INter-SEntential Transformer (2020.acl-main)
Copied to clipboard
| Challenge: | Missing sentence generation fosters a wide range of applications in natural language generation . Developing models for sentence infilling can potentially facilitate many text generation applications . |
| Approach: | They propose a framework to decouple the problem from natural language processing . they propose generating missing sentences that can syntactically and semantically bridge context . |
| Outcome: | The proposed model learns a sentence representation and generates 'missing sentences' the proposed model can be used for document auto-completion and meeting note expansion . |
Sub-Sentence Encoder: Contrastive Learning of Propositional Semantic Representations (2024.naacl-long)
Copied to clipboard
Sihao Chen, Hongming Zhang, Tong Chen, Ben Zhou, Wenhao Yu, Dian Yu, Baolin Peng, Hongwei Wang, Dan Roth, Dong Yu
| Challenge: | Sentence embeddings are typically learned to recognize the semantic relation between two text inputs. |
| Approach: | They introduce a contrastively-learned contextual embedding model for fine-grained semantic representation of text. |
| Outcome: | The proposed model is able to produce contextual embeddings corresponding to different atomic propositions, i.e. semantic equivalence between propositions across different text sequences. |
Adaptive Reinforcement Tuning Language Models as Hard Data Generators for Sentence Representation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods use contrastive learning (CL) to learn effective sentence representations, but require extensive human annotation. |
| Approach: | They propose a reinforcement learning approach for fine-tuning small-parameter LLMs to generate high-quality hard contrastive data without human feedback. |
| Outcome: | The proposed method achieves state-of-the-art on seven semantic text similarity tasks. |
Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models (2022.findings-acl)
Copied to clipboard
| Challenge: | Sentence embeddings are useful for language processing tasks, but it is unclear how to produce them from encoder-decoder models. |
| Approach: | They investigate the effects of scaling up sentence encoders to 11B parameters on sentence embeddings from text-to-text transformers (T5) . |
| Outcome: | The proposed models outperform the previous best models on both SentEval and SentGLUE transfer tasks. |
A Semantically Consistent and Syntactically Variational Encoder-Decoder Framework for Paraphrase Generation (2020.coling-main)
Copied to clipboard
| Challenge: | Paraphrase generation is a longstanding problem in natural language processing (NLP) Neural network-based methods have shown great progress on paraphrase generation. |
| Approach: | They propose a framework that integrates variational inference on a target-related latent variable to introduce the diversity. |
| Outcome: | The proposed framework outperforms baseline models on the metrics based on n-gram matching and semantic similarity, and it can generate multiple different paraphrases by assembling different syntactic variables. |