A Novel Estimator of Mutual Information for Learning to Disentangle Textual Representations (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for learning disentangled representations of textual data are difficult to implement and suffer from the degeneracy of other losses in multi-class scenarios. |
| Approach: | They propose a variational upper bound to the mutual information between an attribute and the latent code of an encoder that controls the approximation error. |
| Outcome: | The proposed method is superior on fair classification and on textual style transfer tasks. |
Similar Papers
Learning Disentangled Textual Representations via Statistical Measures of Similarity (2022.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to disentangle a sensitive attribute from textual representations require training and multiple parameter updates. |
| Approach: | They propose a family of regularizers for learning disentangled representations that do not require training. |
| Outcome: | The proposed regularizers are faster and faster and achieve better results when combined with pretrained and randomly initialized text encoders. |
An Evaluation of Disentangled Representation Learning for Texts (2021.findings-acl)
Copied to clipboard
| Challenge: | Disentangled representations of texts encode information pertaining to different aspects of the text in separate vector embeddings. |
| Approach: | They propose to use a highly-structured natural language dataset to evaluate disentangled representations for texts. |
| Outcome: | The proposed models are well-suited for learning disentangled representations of texts on a synthetic natural language dataset. |
Improving Disentangled Text Representation Learning with Information-Theoretic Guidance (2020.acl-main)
Copied to clipboard
Pengyu Cheng, Martin Renqiang Min, Dinghan Shen, Christopher Malon, Yizhe Zhang, Yitong Li, Lawrence Carin
| Challenge: | Disentangled representation learning (DRL) maps different aspects of data into distinct and independent low-dimensional latent vector spaces. |
| Approach: | They propose a method that manifests disentangled representations of text without supervision on semantics by minimizing the upper bound between style and content. |
| Outcome: | The proposed method improves on conditional text generation and text-style transfer tasks and improves style preservation. |
Learning Disentangled Representations of Negation and Uncertainty (2022.acl-long)
Copied to clipboard
| Challenge: | Negation and uncertainty modeling are long-standing tasks in natural language processing. |
| Approach: | They propose to disentangle negation, uncertainty, and content using a Variational Autoencoder by supervising latent representations using auxiliary objectives. |
| Outcome: | The proposed model can disentangle negation, uncertainty, and content using a Variational Autoencoder. |
Disentangled Representation Learning for Non-Parallel Text Style Transfer (P19-1)
Copied to clipboard
| Challenge: | a paper aims to disentangle latent representations of style and content in language models . auxiliary multi-task and adversarial objectives are used to disentangle the latent space . |
| Approach: | They propose a simple yet effective approach to disentangling latent representations . they propose auxiliary multi-task and adversarial objectives to disentangle style and content . |
| Outcome: | The proposed approach achieves high performance in terms of transfer accuracy, content preservation, and language fluency compared to previous approaches . |
Polarized-VAE: Proximity Based Disentangled Representation Learning for Text Generation (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for learning disentangled representations of real-world data focus on attribute labels or unsupervised methods that manipulate factorization in the latent space of models such as the variational autoencoder (VAE). |
| Approach: | They propose an approach that disentangles select attributes in the latent space based on proximity measures reflecting the similarity between data points with respect to these attributes. |
| Outcome: | The proposed method outperforms the VAE baseline and is competitive with state-of-the-art approaches while being more a general framework applicable to other attribute disentanglement tasks. |
Disentangling Generative Factors in Natural Language with Discrete Variational Autoencoders (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Disentangled representation learning aims to provide an interpretable representation of latent features and a framework for controlling the change of specific features. |
| Approach: | They propose a Variational Autoencoder based method which models language features as discrete variables and encourages independence between variables for learning disentangled representations. |
| Outcome: | The proposed model outperforms baselines on several qualitative and quantitative benchmarks and on a text style transfer downstream application. |
Attribute Alignment: Controlling Text Generation from Pre-trained Language Models (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models can generate text with sentiment polarity or specific topics without changing the original model parameters. |
| Approach: | They propose a method for controlling text generation by aligning disentangled attribute representations. |
| Outcome: | The proposed method shows large performance gains while maintaining diversity and fluency. |
Enhancing Variational Autoencoders with Mutual Information Neural Estimation for Text Generation (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to train variational autoencoders (VAEs) have been proposed to alleviate the posterior collapse issue in NLP tasks. |
| Approach: | They propose to introduce a mutual information term between the input and its latent variable to regularize the objective of the VAE. |
| Outcome: | The proposed model performs better on three benchmark datasets and is comparable to state-of-the-art models. |
Learning Disentangled Representations for Natural Language Definitions (2023.findings-eacl)
Copied to clipboard
| Challenge: | Disentangling the encodings of neural models is a fundamental aspect for improving interpretability, semantic control and downstream task performance in Natural Language Processing. |
| Approach: | They propose to use syntactic and semantic regularities in textual data to provide models with both structural biases and generative factors. |
| Outcome: | The proposed model outperforms baselines on several qualitative and quantitative benchmarks and improves the results in the downstream task of definition modeling. |