Plug-in Language Model: Controlling Text Generation with a Simple Regression Model (2024.findings-naacl)
Copied to clipboard
| Challenge: | Large-scale pre-trained language models have demonstrated unrivaled capacity in generating text that closely resembles human-written content. |
| Approach: | They propose a plug-in language model that leverages reinforcement learning to adjust latent states to control text generation. |
| Outcome: | The proposed model outperforms existing methods that rely on gradient-based, weighted decoding, or prompt-based methods. |
Similar Papers
A Plug-and-Play Method for Controlled Text Generation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for controlling language generation are not able to produce fluent text . current methods require additional models or fine-tuning to ensure specific words are included . |
| Approach: | They propose a plug-and-play decoding method that allows for controlled language generation . they add a shift in the probability distribution over our vocabulary towards semantically similar words . |
| Outcome: | The proposed method outperforms competing methods in human evaluations and does not impact fluency. |
Fine-Grained Controllable Text Generation Using Non-Residual Prompting (2022.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to control the text generation process are not expressive enough. |
| Approach: | They propose an encoder-decoder architecture that enables intermediate text prompts at arbitrary time steps. |
| Outcome: | The proposed architecture is expressive and versatile on multiple experimental settings. |
Improving the Language Understanding Capabilities of Large Language Models Using Reinforcement Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Instruction-fine-tuned large language models (LLMs) under 14B parameters underperform on NLU tasks . we explore a framework to improve the NLU capabilities of LLMs . |
| Approach: | They propose to use Proximal Policy Optimization to improve NLU capabilities . they frame NLU as a reinforcement learning environment and optimize for reward signals . |
| Outcome: | The proposed framework outperforms supervised fine-tuning on GLUE and superGLUE tasks. |
Continual Reinforcement Learning for Controlled Text Generation (2024.lrec-main)
Copied to clipboard
| Challenge: | Controlled Text Generation (CTG) aims to steer text generation towards texts possessing a desired attribute. |
| Approach: | They propose an algorithm that steers the generation of continuations of a given context . they propose a Continual Learning problem to learn at every step to steer next-word generation . |
| Outcome: | The proposed algorithm is based on a plug-and-play language model and exhibits promising results. |
CFL: Causally Fair Language Models Through Token-level Attribute Controlled Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to control attributes of Language Models (LMs) for text generation are not safe, as toxicity and bias goals are opposed to each other. |
| Approach: | They propose a method to control the attributes of Language Models (LMs) for the text generation task using Causal Average Treatment Effect (ATE) scores and counterfactual augmentation. |
| Outcome: | The proposed architecture achieves state of the art performance for toxic degeneration, which are computed using Real Toxicity Prompts. |
Exploring Versatile Generative Language Model Via Parameter-Efficient Transfer Learning (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Large-scale language models can be fine-tuned to learn highly transferable embedding, but they are expensive and require multiple model parameters. |
| Approach: | They propose a way to fine-tune multiple down-stream generation tasks simultaneously using a single, large pretrained model. |
| Outcome: | The proposed model can maintain or improve the performance of fine-tuning the whole model. |
Adaptive Reinforcement Tuning Language Models as Hard Data Generators for Sentence Representation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods use contrastive learning (CL) to learn effective sentence representations, but require extensive human annotation. |
| Approach: | They propose a reinforcement learning approach for fine-tuning small-parameter LLMs to generate high-quality hard contrastive data without human feedback. |
| Outcome: | The proposed method achieves state-of-the-art on seven semantic text similarity tasks. |
CDLM: Cross-Document Language Modeling (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing language models (LMs) provide powerful representations for internal text structure, but there are important applications for multi-text tasks. |
| Approach: | They propose a pretraining approach that incorporates two key ideas into the masked language modeling objective. |
| Outcome: | The proposed model improves over existing models and sets of long-range transformers and can be easily applied to multiple multi-text tasks. |
Reinforcement Learning with Token-level Feedback for Controllable Text Generation (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for controllable text generation are guided by coarse-grained feedback, which may lead to suboptimal performance owing to semantic twists or progressions within sentences. |
| Approach: | They propose a reinforcement learning algorithm which formulates TOken-LEvel rewards for controllable text generation and employs a "first-quantize-then-noise" paradigm to enhance the robustness of the RL algorithm. |
| Outcome: | The proposed algorithm can achieve superior performance on single-attribute and multi-attract control tasks. |
Open-ended Long Text Generation via Masked Language Modeling (2023.acl-long)
Copied to clipboard
| Challenge: | Pre-trained autoregressive language models have dominated OPen-ended Long Text Generation (Open-LTG) however, the low inference efficiency of AR impedes their usability. |
| Approach: | They propose a representative iterative non-autoregressive (NAR) decoding strategy to improve inference efficiency for Open-LTG. |
| Outcome: | The proposed model can generate short text and collapse for long text modeling. |