A Deep Decomposable Model for Disentangling Syntax and Semantics in Sentence Representation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in disentanglement work on coarse levels in the disenanglement of closely related properties, such as syntax and semantics in human languages. |
| Approach: | They propose a deep decomposable model based on VAE to disentangle syntax and semantics by using total correlation penalties on KL divergences. |
| Outcome: | The proposed model significantly improves the disentanglement quality between syntactic and semantic representations for semantic similarity tasks and syntaktic similarity task. |
Similar Papers
A Multi-Task Approach for Disentangling Syntax and Semantics in Sentence Representations (N19-1)
Copied to clipboard
| Challenge: | Empirically, the model with the best performing syntactic and semantic representations gives rise to the most disentangled representations. |
| Approach: | They propose a generative model that uses latent variables to learn a sentence that uses both latent and latent representations. |
| Outcome: | The proposed model achieves better disentanglement between semantic and syntactic representations by training with multiple losses, including losses that exploit aligned paraphrastic sentences and word-order information. |
Learning Disentangled Representations for Natural Language Definitions (2023.findings-eacl)
Copied to clipboard
| Challenge: | Disentangling the encodings of neural models is a fundamental aspect for improving interpretability, semantic control and downstream task performance in Natural Language Processing. |
| Approach: | They propose to use syntactic and semantic regularities in textual data to provide models with both structural biases and generative factors. |
| Outcome: | The proposed model outperforms baselines on several qualitative and quantitative benchmarks and improves the results in the downstream task of definition modeling. |
Polarized-VAE: Proximity Based Disentangled Representation Learning for Text Generation (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for learning disentangled representations of real-world data focus on attribute labels or unsupervised methods that manipulate factorization in the latent space of models such as the variational autoencoder (VAE). |
| Approach: | They propose an approach that disentangles select attributes in the latent space based on proximity measures reflecting the similarity between data points with respect to these attributes. |
| Outcome: | The proposed method outperforms the VAE baseline and is competitive with state-of-the-art approaches while being more a general framework applicable to other attribute disentanglement tasks. |
Learning Disentangled Representations of Negation and Uncertainty (2022.acl-long)
Copied to clipboard
| Challenge: | Negation and uncertainty modeling are long-standing tasks in natural language processing. |
| Approach: | They propose to disentangle negation, uncertainty, and content using a Variational Autoencoder by supervising latent representations using auxiliary objectives. |
| Outcome: | The proposed model can disentangle negation, uncertainty, and content using a Variational Autoencoder. |
Exploiting Inductive Bias in Transformers for Unsupervised Disentanglement of Syntax and Semantics with VAEs (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing models for text generation do not need syntactic information such as constituency parses or semantic information such a paraphrase pairs. |
| Approach: | They propose a generative model which exhibits disentangled latent representations of syntax and semantics by using Attention in its decoder. |
| Outcome: | The proposed model outperforms supervised models on syntax/semantics transfer and shows that it can read latent variables with keys and values. |
Explaining Word Embeddings via Disentangled Representation (2020.aacl-main)
Copied to clipboard
| Challenge: | Disentangled representations are known to represent interpretable factors in separated dimensions. |
| Approach: | They propose to transform dense word vectors into disentangled embeddings with improved interpretability by encoding polysemous semantics separately. |
| Outcome: | The proposed model can be encoded into multiple sub-embeddings or sub-areas and generates more efficient and effective features for natural language processing. |
VAEGPT-Sim: Improving Sentence Representation with Limited Corpus Using Gradually-Denoising VAE (2024.findings-acl)
Copied to clipboard
| Challenge: | Text embedding requires a highly efficient method for training domain-specific models on limited corpora. |
| Approach: | They propose a model that combines a denoising variational autoencoder with a target-specific discriminator to generate synonymous sentences that closely resemble human language. |
| Outcome: | The proposed model surpasses ConSERT by 2.8 points in small-dataset training on STS benchmarks. |
Learning Disentangled Semantic Spaces of Explanations via Invertible Neural Networks (2024.acl-long)
Copied to clipboard
| Challenge: | Disentangled latent spaces usually have better semantic separability and geometrical properties, which leads to better interpretability and controllable data generation. |
| Approach: | They propose a flow-based invertible neural network mechanism integrated with a transformer-based language Autoencoder to deliver latent spaces with better semantic separability and geometrical properties. |
| Outcome: | The proposed model can deliver latent spaces with better separability properties compared to the current state-of-the-art models. |
Generating Sentences from Disentangled Syntactic and Semantic Spaces (P19-1)
Copied to clipboard
| Challenge: | Variational auto-encoders (VAEs) are widely used in natural language generation due to the regularization of the latent space. |
| Approach: | They propose to generate sentences from disentangled syntactic and semantic spaces by using the linearized tree sequence. |
| Outcome: | The proposed method achieves similar or better performance in various tasks compared with state-of-the-art models. |
Improving Text Understanding via Deep Syntax-Semantics Communication (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that integrating syntactic tree models with sequential semantic models can bring improved task performance. |
| Approach: | They propose a deep neural communication model between syntax and semantics to improve the performance of text understanding. |
| Outcome: | The proposed model outperforms baseline models on syntax-dependent tasks by a large margin. |