A Multi-Task Approach for Disentangling Syntax and Semantics in Sentence Representations (N19-1)
Copied to clipboard
| Challenge: | Empirically, the model with the best performing syntactic and semantic representations gives rise to the most disentangled representations. |
| Approach: | They propose a generative model that uses latent variables to learn a sentence that uses both latent and latent representations. |
| Outcome: | The proposed model achieves better disentanglement between semantic and syntactic representations by training with multiple losses, including losses that exploit aligned paraphrastic sentences and word-order information. |
Similar Papers
A Deep Decomposable Model for Disentangling Syntax and Semantics in Sentence Representation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in disentanglement work on coarse levels in the disenanglement of closely related properties, such as syntax and semantics in human languages. |
| Approach: | They propose a deep decomposable model based on VAE to disentangle syntax and semantics by using total correlation penalties on KL divergences. |
| Outcome: | The proposed model significantly improves the disentanglement quality between syntactic and semantic representations for semantic similarity tasks and syntaktic similarity task. |
Learning Disentangled Representations for Natural Language Definitions (2023.findings-eacl)
Copied to clipboard
| Challenge: | Disentangling the encodings of neural models is a fundamental aspect for improving interpretability, semantic control and downstream task performance in Natural Language Processing. |
| Approach: | They propose to use syntactic and semantic regularities in textual data to provide models with both structural biases and generative factors. |
| Outcome: | The proposed model outperforms baselines on several qualitative and quantitative benchmarks and improves the results in the downstream task of definition modeling. |
Disentangling Semantics and Syntax in Sentence Embeddings with Pre-trained Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Pre-trained language models have been successful on a wide range of NLP tasks . however, contextual representations from pre-trated models contain entangled semantic and syntactic information. |
| Approach: | They propose a semantic sentence embedding model that disentangles semantics and syntax from pre-trained models. |
| Outcome: | The proposed model outperforms state-of-the-art models on unsupervised semantic similarity tasks. |
Disentangling Meaning and Language Components in Diverse Multilingual Sentence Embeddings (2026.acl-srw)
Copied to clipboard
| Challenge: | Existing studies have reported language specificity in multilingual sentence embeddings, resulting in language-specific subspaces. |
| Approach: | They propose to disentangle multilingual sentence embeddings into language-dependent and language-agnostic components to improve cross-lingual similarity estimation. |
| Outcome: | The proposed methods improve cross-lingual similarity estimation across multiple embeddings. |
Semantically Driven Sentence Fusion: Modeling and Evaluation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Sentence fusion is the task of joining related sentences into coherent text. |
| Approach: | They propose a method where ground-truth solutions are automatically expanded into multiple references via curated equivalence classes of connective phrases. |
| Outcome: | The proposed approach improves on state-of-the-art models by expanding ground-truth solutions into multiple references. |
Extending Multi-Text Sentence Fusion Resources via Pyramid Annotations (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing datasets for sentence fusion tasks are limited in size and scope . despite recent advances, cross-document tasks such as multi-document summarization have not progressed with the same pace. |
| Approach: | They propose to extend a sentence fusion dataset by almost four times its original size . they relabel the dataset and employ more data sources to improve model performance . |
| Outcome: | The proposed dataset triples the size of an earlier dataset and improves performance . it also includes more complex training instances better reflecting those found in "the wild" |
Explaining Word Embeddings via Disentangled Representation (2020.aacl-main)
Copied to clipboard
| Challenge: | Disentangled representations are known to represent interpretable factors in separated dimensions. |
| Approach: | They propose to transform dense word vectors into disentangled embeddings with improved interpretability by encoding polysemous semantics separately. |
| Outcome: | The proposed model can be encoded into multiple sub-embeddings or sub-areas and generates more efficient and effective features for natural language processing. |
What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties (P18-1)
Copied to clipboard
| Challenge: | a lack of understanding of the properties of sentence embeddings is limiting the use of the techniques. |
| Approach: | They propose 10 probing tasks designed to capture simple linguistic features of sentences . they use three different encoders to train embeddings in eight different ways . |
| Outcome: | The proposed tasks capture key linguistic features of sentences, but they are difficult to infer from them. |
A Bilingual Generative Transformer for Semantic Sentence Embedding (2020.emnlp-main)
Copied to clipboard
| Challenge: | Semantic sentence embedding models encode natural language sentences into vectors, such that closeness in embeddable space indicates closeness of semantics between the sentences. |
| Approach: | They propose a deep latent variable model that attempts to perform source separation on parallel sentences, isolating what they have in common in a latent semantic vector, and explaining what is left over with language-specific latent vectors. |
| Outcome: | The proposed model outperforms the state-of-the-art on a standard suite of unsupervised semantic similarity evaluations. |
Controllable Paraphrase Generation with a Syntactic Exemplar (P19-1)
Copied to clipboard
| Challenge: | Prior work on controllable text generation assumes that the generated attribute can take on a finite set of values known a priori. |
| Approach: | They propose a task where the syntax of a generated sentence is controlled rather by a sentential exemplar. |
| Outcome: | The proposed model achieves improvements over baselines and learns to capture desirable characteristics. |