Continual Reinforcement Learning for Controlled Text Generation (2024.lrec-main)
Copied to clipboard
| Challenge: | Controlled Text Generation (CTG) aims to steer text generation towards texts possessing a desired attribute. |
| Approach: | They propose an algorithm that steers the generation of continuations of a given context . they propose a Continual Learning problem to learn at every step to steer next-word generation . |
| Outcome: | The proposed algorithm is based on a plug-and-play language model and exhibits promising results. |
Similar Papers
Reinforcement Learning with Token-level Feedback for Controllable Text Generation (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for controllable text generation are guided by coarse-grained feedback, which may lead to suboptimal performance owing to semantic twists or progressions within sentences. |
| Approach: | They propose a reinforcement learning algorithm which formulates TOken-LEvel rewards for controllable text generation and employs a "first-quantize-then-noise" paradigm to enhance the robustness of the RL algorithm. |
| Outcome: | The proposed algorithm can achieve superior performance on single-attribute and multi-attract control tasks. |
Plug-in Language Model: Controlling Text Generation with a Simple Regression Model (2024.findings-naacl)
Copied to clipboard
| Challenge: | Large-scale pre-trained language models have demonstrated unrivaled capacity in generating text that closely resembles human-written content. |
| Approach: | They propose a plug-in language model that leverages reinforcement learning to adjust latent states to control text generation. |
| Outcome: | The proposed model outperforms existing methods that rely on gradient-based, weighted decoding, or prompt-based methods. |
Fine-tuned Language Models are Continual Learners (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on large language models relies on intuition that most tasks can be described via natural language instructions. |
| Approach: | They propose that a model should be able to keep extending its knowledge without forgetting previous skills. |
| Outcome: | The proposed model can learn 8 new diverse language generation tasks while maintaining good performance on previous tasks, spanning in total of 70 datasets. |
Language Model Sentence Completion with a Parser-Driven Rhetorical Control Method (2024.eacl-short)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) generate nonfacts and deviate from desired criteria for text generation. |
| Approach: | They propose a controlled text generation algorithm that enforces adherence toward specific rhetorical relations in an LLM sentence-completion context by a parser-driven decoding scheme. |
| Outcome: | The proposed method generates sentences that satisfy desired rhetorical relations in an LLM. |
Controlled Text Generation for Large Language Model with Dynamic Attribute Graphs (2024.findings-acl)
Copied to clipboard
| Challenge: | Controlled Text Generation (CTG) aims to produce texts that exhibit specific desired attributes. |
| Approach: | They propose a pluggable CTG framework for Large Language Models to control text . they use attribute scorers to evaluate attributes of sentences and construct dynamic attribute graphs . |
| Outcome: | The proposed framework achieves a peak improvement of 19.29% over baseline methods in two tasks. |
Continual Learning for Natural Language Generation in Task-oriented Dialog Systems (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing neural approaches for natural language generation are typically developed offline for specific domains. |
| Approach: | They propose a method to expand NLG knowledge incrementally to new domains . major challenge is catastrophic forgetting, meaning a model forgets the knowledge it has learned before . |
| Outcome: | The proposed method outperforms other methods by effectively mitigating catastrophic forgetting issue. |
Rethinking Long Context Generation from the Continual Learning Perspective (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) struggle with processing long contexts due to the limited context window. |
| Approach: | They propose to combine a limited context window with a continual learning perspective to improve LLMs' efficiency in processing long contexts. |
| Outcome: | The proposed models improve the performance of Large Language Models (LLMs) by integrating learning strategies with existing approaches. |
Unlocking Anticipatory Text Generation: A Constrained Approach for Large Language Models Decoding (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models have shown a powerful ability for text generation, but undesired behaviors such as toxicity and hallucinations can manifest. |
| Approach: | They propose to formalize text generation as a future-constrained generation problem to minimize undesirable behaviors and enforce faithfulness to instructions. |
| Outcome: | The proposed approach is effective across three tasks, including keyword-constrained generation, toxicity reduction, and factual correctness in question-answering. |
Controllable Text Generation with Residual Memory Transformer (2024.findings-acl)
Copied to clipboard
| Challenge: | Large-scale Causal Language Models (CLMs) have been successful in text generation, but there is still a challenge to control the generation process. |
| Approach: | They propose a non-intrusive, lightweight control plugin to control the generation process of a CLM at arbitrary time steps. |
| Outcome: | The proposed plugin can handle any type of control conditions and cooperate with the base CLM through a residual learning paradigm. |
CIE: Controlling Language Model Text Generations Using Continuous Signals (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to control language models with intent are brittle and hard to scale. |
| Approach: | They propose to use a set of LMs to fine-tune to expect a control vector that is interpolated between a "low" and a 'high' token embedding. |
| Outcome: | The proposed method can be finetuned to expect a control vector that is interpolated between a “low” and a ‘high” token embedding. |