FreeCtrl: Constructing Control Centers with Feedforward Layers for Learning-Free Controllable Text Generation (2024.acl-long)
Copied to clipboard
| Challenge: | Controllable text generation (CTG) focuses on crafting texts adhering to specific attributes . studies show learning-based methods require extensive computational and data resources . |
| Approach: | They propose a learning-free approach that dynamically adjusts the weights of selected feedforward neural network vectors to steer the outputs of large language models. |
| Outcome: | The proposed approach outperforms learning-based and learning-free methods on multi-attribute control. |
Similar Papers
RSA-Control: A Pragmatics-Grounded Lightweight Controllable Text Generation Framework (2024.emnlp-main)
Copied to clipboard
| Challenge: | RSA-Control is a training-free controllable text generation framework . existing studies rely on fine-tuning pre-trained language models . external components could hurt coherence and accuracy of the model . |
| Approach: | They propose a training-free controllable text generation framework grounded in pragmatics that directs the generation process by recursively reasoning between imaginary speakers and listeners. |
| Outcome: | The proposed framework achieves strong attribute control while maintaining fluency and content consistency. |
Focused Prefix Tuning for Controllable Text Generation (2023.acl-short)
Copied to clipboard
| Challenge: | Existing unannotated attributes could degrade models' performance . focus on the desired attribute can be achieved with focused prefix tuning . |
| Approach: | They propose focused prefix tuning to enable the control to focus on the desired attribute . they propose to reduce the number of unannotated attributes in a controllable text generation dataset . |
| Outcome: | The proposed approach achieves better control accuracy and text fluency than baseline models in single-attribute tasks. |
CEV-LM: Controlled Edit Vector Language Model for Shaping Natural Language Generations (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing control approaches primarily adjust the semantic (e.g., emotion, topics), structural (e-speech, parts-of-seech), and lexical (el-s-sp-s) properties of text, but are insufficient to accomplish complex objectives such as pacing which control the complexity and readability of the text. |
| Approach: | They propose a lightweight semi-autoregressive language model that uses edit vectors to control three complementary metrics that quantify the shape of text. |
| Outcome: | The proposed model provides significantly more targeted and precise control of speed, volume, and circuitousness while using less training data, and containing fewer parameters. |
CTRL: Control-Based Time Series Forecasting with LLM-Guided Residual Learning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing time series forecasting approaches reduce them to numerical predictors that bypass their strengths or allow direct forecast generation that destabilizes predictions in non-stationary settings. |
| Approach: | They propose a framework that decouples semantic reasoning from quantitative prediction. |
| Outcome: | The proposed framework decouples semantic reasoning from quantitative prediction. |
Air-Decoding: Attribute Distribution Reconstruction for Decoding-Time Controllable Text Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Controllable text generation (CTG) aims to generate text with desired attributes, but current methods lack high levels of controllability. |
| Approach: | They propose a lightweight decoding framework that reconstructs attribute distributions to balance the weights between attribute words and non-attribute words to generate more fluent text. |
| Outcome: | The proposed framework achieves state-of-the-art control performance on multiple CTG tasks. |
Mix and Match: Learning-free Controllable Text Generationusing Energy Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | Recent work on controlled text generation has required attribute-based fine-tuning of the base language model or restricted the parameterization of the attribute discriminator. |
| Approach: | They propose a global score-based alternative for controllable text generation that combines arbitrary pre-trained black-box models for achieving desired attributes in the generated text. |
| Outcome: | The proposed method outperforms methods that require extra training or fine-tuning . the proposed method is based on a model with energy values of a linear combination of scores from black-box models . |
Exploring Controllable Text Generation Techniques (2020.coling-main)
Copied to clipboard
| Challenge: | Neural controllable text generation has a plethora of applications but there is no unifying theme. |
| Approach: | They propose a new schema for the control of attributes in the generation process by classifying it into five modules and providing an analysis on the advantages and disadvantages of these techniques. |
| Outcome: | The proposed frameworks can be used to control the attributes of natural sentences and to modulate the formality and politeness of emails. |
CTRLEval: An Unsupervised Reference-Free Metric for Evaluating Controlled Text Generation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing reference-free metrics have obvious limitations for evaluating controlled text generation models. |
| Approach: | They propose an unsupervised reference-free metric which evaluates controlled text generation from different aspects by formulating each aspect into multiple text infilling tasks. |
| Outcome: | The proposed metric has higher correlations with human judgments while obtaining better generalization of evaluating generated texts from different models and with different qualities. |
Reinforcement Learning with Token-level Feedback for Controllable Text Generation (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for controllable text generation are guided by coarse-grained feedback, which may lead to suboptimal performance owing to semantic twists or progressions within sentences. |
| Approach: | They propose a reinforcement learning algorithm which formulates TOken-LEvel rewards for controllable text generation and employs a "first-quantize-then-noise" paradigm to enhance the robustness of the RL algorithm. |
| Outcome: | The proposed algorithm can achieve superior performance on single-attribute and multi-attract control tasks. |
Tailor: A Soft-Prompt-Based Approach to Attribute-Based Controlled Text Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing work focuses on generating sentences satisfying pre-specified attributes such as topic and sentiment, yet suffers from increases in storage and inference time. |
| Approach: | They propose a method that uses a pre-trained continuous vector to generate a fixed pre-trainable language model to satisfy a specified attribute. |
| Outcome: | The proposed model can achieve improvements on eleven attribute-specific generation tasks with 0.08% extra training parameters. |