Attribute Alignment: Controlling Text Generation from Pre-trained Language Models (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models can generate text with sentiment polarity or specific topics without changing the original model parameters. |
| Approach: | They propose a method for controlling text generation by aligning disentangled attribute representations. |
| Outcome: | The proposed method shows large performance gains while maintaining diversity and fluency. |
Similar Papers
Multi-Attribute Controlled Text Generation with Contrastive-Generator and External-Discriminator (2022.coling-1)
Copied to clipboard
| Challenge: | Existing studies on controlled text generation focus on single-attribute control, but in practical applications, they lack controllability. |
| Approach: | They propose a framework for multi-attribute controlled text generation that can effectively generate texts with more attributes. |
| Outcome: | The proposed framework achieves remarkable controllability while keeping the text fluent and diverse. |
An Invariant Learning Characterization of Controlled Text Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Controlled generation is a problem of creating text that contains stylistic or semantic attributes of interest. |
| Approach: | They propose a distribution shift-based control system that can be used to train a predictor of the desired attribute. |
| Outcome: | The proposed method shows that the most effective predictor should be invariant across multiple text environments. |
CAT-Gen: Improving Robustness in NLP Models via Controlled Adversarial Text Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing adversarial text generation approaches can lead to generation lacking diversity or fluency, whereas perturbing in the intermediate representation space can lead a model to generate generations that are not related to the input. |
| Approach: | They propose to generate adversarial texts through controllable attributes that are known to be invariant to task labels. |
| Outcome: | The proposed model generates more diverse and fluent adversarial examples, compared to existing approaches, and is more robust against model re-training and different model architectures. |
Tailor: A Soft-Prompt-Based Approach to Attribute-Based Controlled Text Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing work focuses on generating sentences satisfying pre-specified attributes such as topic and sentiment, yet suffers from increases in storage and inference time. |
| Approach: | They propose a method that uses a pre-trained continuous vector to generate a fixed pre-trainable language model to satisfy a specified attribute. |
| Outcome: | The proposed model can achieve improvements on eleven attribute-specific generation tasks with 0.08% extra training parameters. |
Partially-Aligned Data-to-Text Generation with Distant Supervision (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using partially-aligned data is an alternative way of solving the dataset scarcity problem. |
| Approach: | They propose a task to generate human-readable text for describing some given structured data enabling more interpretability. |
| Outcome: | The proposed framework outperforms baseline models and validates the feasibility of using partially-aligned data. |
Exploring Controllable Text Generation Techniques (2020.coling-main)
Copied to clipboard
| Challenge: | Neural controllable text generation has a plethora of applications but there is no unifying theme. |
| Approach: | They propose a new schema for the control of attributes in the generation process by classifying it into five modules and providing an analysis on the advantages and disadvantages of these techniques. |
| Outcome: | The proposed frameworks can be used to control the attributes of natural sentences and to modulate the formality and politeness of emails. |
A Distributional Lens for Multi-Aspect Controllable Text Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for multi-aspect control suffer from attribute degeneration due to mutual interference of these controllers. |
| Approach: | They propose to use attribute fusion to find the intersections of multiple attributes as their combination for generation. |
| Outcome: | The proposed method outperforms baselines on attribute relevance and text quality and achieves the SOTA. |
Controlled Text Generation with Hidden Representation Transformations (2023.findings-acl)
Copied to clipboard
| Challenge: | Using a con-trolled language model, we gain attribute control by modifying the hidden representation of thebase model through learning transformations. |
| Approach: | They propose a con-trolled language generation framework that gains attribute control bymodifying the hidden representation of thebase model through learned transformations. |
| Outcome: | The proposed framework outperforms all thebaselines in detoxification, positivesentiment steering, and text simplification while minimizing the loss in linguistic qualities. |
Controlled Language Generation for Language Learning Items (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Recent advances in pre-trained language models have resulted in success in generating fluent English text. |
| Approach: | They propose to employ natural language generation to rapidly generate English language items . they experiment with deep pretrained models and develop methods for controlling items for factors relevant in language learning . |
| Outcome: | The proposed framework shows high grammatically scores for all models and higher complexity over baseline models. |
TARA: Token-level Attribute Relation Adaptation for Multi-Attribute Controllable Text Generation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on multi-attribute controllable text generation ignores interrelations of attributes . recent work defines attribute relations as promotive, but not fixed . |
| Approach: | They propose a method that explicitly defines attribute relations as inhibtory for multi-attribute CTG . they propose 'tara' which employs token-level attribute relation adaptation and representation to generate text with the balanced multi-attribut . |
| Outcome: | The proposed method generates text with the balanced multi-attribute control. |