DisCup: Discriminator Cooperative Unlikelihood Prompt-tuning for Controllable Text Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing prompt tuning approaches for attribute-controllable text generation are difficult to implement due to the lack of interpretability of deep neural networks. |
| Approach: | They propose a new approach that incorporates attribute knowledge of discriminator to optimize prompt tuning by steering a frozen CLM to produce attribute-specific texts. |
| Outcome: | The proposed approach can achieve state-of-the-art control performance while maintaining high-quality text generation. |
Similar Papers
Tailor: A Soft-Prompt-Based Approach to Attribute-Based Controlled Text Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing work focuses on generating sentences satisfying pre-specified attributes such as topic and sentiment, yet suffers from increases in storage and inference time. |
| Approach: | They propose a method that uses a pre-trained continuous vector to generate a fixed pre-trainable language model to satisfy a specified attribute. |
| Outcome: | The proposed model can achieve improvements on eleven attribute-specific generation tasks with 0.08% extra training parameters. |
Prompt Tuning for Discriminative Pre-trained Language Models (2022.findings-acl)
Copied to clipboard
Yuan Yao, Bowen Dong, Ao Zhang, Zhengyan Zhang, Ruobing Xie, Zhiyuan Liu, Leyu Lin, Maosong Sun, Jianyong Wang
| Challenge: | Recent studies have shown promising results of prompt tuning in stimulating pre-trained language models (PLMs) for natural language processing tasks. |
| Approach: | They propose a prompt tuning framework that reformulates NLP tasks into a discriminative language modeling problem. |
| Outcome: | The proposed framework improves on text classification and question answering tasks and prevents unstable tuning problems in low-resource settings. |
FUDGE: Controlled Text Generation With Future Discriminators (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent advances in large pretrained language models allow us to generate increasingly realistic text by modeling a distribution P (X) over natural language sequences X. |
| Approach: | They propose a flexible and modular method for controlled text generation that uses a Bayesian decomposition of the conditional distribution of G given an attribute predictor and can easily compose predictors for multiple desired attributes. |
| Outcome: | The proposed method can be easily composed and performs three tasks. |
Attributes as Textual Genes: Leveraging LLMs as Genetic Algorithm Simulators for Conditional Synthetic Data Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Genetic Prompt combines genetic algorithms with Large Language Models to augment synthetic data generation. |
| Approach: | They propose a framework that combines genetic algorithms with LLMs to augment synthetic data generation. |
| Outcome: | The proposed framework outperforms state-of-the-art models and shows robust performance across generator models. |
Controlled Text Generation for Large Language Model with Dynamic Attribute Graphs (2024.findings-acl)
Copied to clipboard
| Challenge: | Controlled Text Generation (CTG) aims to produce texts that exhibit specific desired attributes. |
| Approach: | They propose a pluggable CTG framework for Large Language Models to control text . they use attribute scorers to evaluate attributes of sentences and construct dynamic attribute graphs . |
| Outcome: | The proposed framework achieves a peak improvement of 19.29% over baseline methods in two tasks. |
TARA: Token-level Attribute Relation Adaptation for Multi-Attribute Controllable Text Generation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on multi-attribute controllable text generation ignores interrelations of attributes . recent work defines attribute relations as promotive, but not fixed . |
| Approach: | They propose a method that explicitly defines attribute relations as inhibtory for multi-attribute CTG . they propose 'tara' which employs token-level attribute relation adaptation and representation to generate text with the balanced multi-attribut . |
| Outcome: | The proposed method generates text with the balanced multi-attribute control. |
Control Large Language Models via Divide and Conquer (2024.emnlp-main)
Copied to clipboard
| Challenge: | Lexically Constrained Generation (LCG) is a crucial task of text generation. |
| Approach: | They propose a Divide and Conquer Generation strategy to enhance LLMs' performance in Lexically Constrained Generation with prompt-based controlling. |
| Outcome: | The proposed strategy shows 90% improvement on the most challenging LCG task. |
UCTG: A Unified Controllable Text Generation Framework for Query Auto-Completion (2025.coling-industry)
Copied to clipboard
| Challenge: | Existing approaches to control text generation (CTG) are essentially challenging to adapt to various control objectives and constraints, which results in mixed success. |
| Approach: | They propose a unified controllable text generation framework which integrates a control module, a prompt module, and a generation module. |
| Outcome: | The proposed framework significantly improves query accuracy and coherence in tasks with different objectives and constraints. |
The Power of Scale for Parameter-Efficient Prompt Tuning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Unlike discrete text prompts used by GPT-3, soft prompts are learned through backpropagation and can be tuned to incorporate signals from any number of labeled examples. |
| Approach: | They propose a mechanism for learning "soft prompts" to condition frozen language models to perform specific downstream tasks. |
| Outcome: | The proposed method outperforms fewshot learning using GPT-3 and matches the quality of model tuning as models exceed billions of parameters. |
Mixture of Soft Prompts for Controllable Data Generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) generate fluent text when the target output follows natural language patterns. |
| Approach: | They propose a method that uses large language models to generate fluent text from a limited ontology rather than direct prediction by using soft prompts. |
| Outcome: | The proposed method produces diverse and natural text while preserving label semantics. |