Accelerated Reinforcement Learning for Sentence Generation by Vocabulary Prediction (N19-1)
Copied to clipboard
| Challenge: | In reinforcement learning-based sentence generation, the large action space is often too computa-tionally demanding to be used with large training data. |
| Approach: | They propose to reduce the action space by using dynamic vocabulary prediction to generate a fixed-size small vocabulary for each input to generate its target sentence. |
| Outcome: | The proposed method achieves faster reinforcement learning (2.7x faster) with less GPU memory (2.3x less) and more rewards with fewer iterations of supervised pre-training. |
Similar Papers
Learning to Selectively Learn for Weakly Supervised Paraphrase Generation with Model-based Reinforcement Learning (2022.naacl-main)
Copied to clipboard
| Challenge: | Paraphrase generation is an important natural language generation task . however, the effectiveness of paraphrase generation can be limited due to the limited data available. |
| Approach: | They propose a weakly supervised approach to paraphrase generation that leverages reinforcement learning for effective model training with data selection. |
| Outcome: | The proposed model improves the state-of-the-art performance on four weakly supervised paraphrase generation tasks. |
Reinforcement Learning with Large Action Spaces for Neural Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Recent work has argued that the gains produced by Reinforcement learning are mostly due to promoting tokens that have already received a fairly high probability in pre-training. |
| Approach: | They hypothesize that the large action space is a main obstacle to RL’s effectiveness in MT by reducing the size of the vocabulary without changing the vocabulary. |
| Outcome: | The proposed method improves by 1.5 BLEU points on average. |
Phrase-Level Action Reinforcement Learning for Neural Dialog Response Generation (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for dialog agent training lack a robust action space for entangled information, which can cause bias and deviate from natural human language. |
| Approach: | They propose phrase-level action reinforcement learning which allows the model to alter the sentence structure and content with the sequential action selection. |
| Outcome: | The proposed model achieves competitive results with state-of-the-art models on the MultiWOZ dataset, indicating that it is effective for solving task-oriented dialogs. |
Neural Keyphrase Generation via Reinforcement Learning with Adaptive Rewards (P19-1)
Copied to clipboard
| Challenge: | Existing generative models generate too few keyphrases, but they often generate too many . et al. (2017) propose a reinforcement learning approach for keyphrase generation . |
| Approach: | They propose a reinforcement learning approach that encourages a model to generate sufficient keyphrases with an adaptive reward function. |
| Outcome: | The proposed method improves state-of-the-art generative models with conventional and new evaluation methods on real-world datasets. |
Improving Reinforcement Learning Based Image Captioning with Natural Language Prior (D18-1)
Copied to clipboard
| Challenge: | Recent research shows that Reinforcement Learning (RL) approaches suffer from the exposure bias problem. |
| Approach: | They propose a Reinforcement Learning (RL) based training framework that constrains the action space using an n-gram language prior. |
| Outcome: | The proposed model is more human readable and graceful. |
Efficient Unsupervised Sentence Compression by Fine-tuning Transformers with Reinforcement Learning (2022.acl-long)
Copied to clipboard
| Challenge: | Recent unsupervised sentence compression approaches use custom objectives to guide discrete search, but guided search is expensive at inference time. |
| Approach: | They propose to use reinforcement learning to train effective sentence compression models that are also fast when generating predictions. |
| Outcome: | The proposed model outperforms other unsupervised models while being faster at inference time. |
Generation with Dynamic Vocabulary (2024.emnlp-main)
Copied to clipboard
| Challenge: | Using static vocabulary, vocabulary is ignored in advanced generation tasks. |
| Approach: | They propose a dynamic vocabulary that can involve arbitrary text spans during generation. |
| Outcome: | The proposed vocabulary can be deployed in a plug-and-play way, thus is attractive for various downstream applications. |
Controllable Text Simplification with Deep Reinforcement Learning (2022.aacl-short)
Copied to clipboard
| Challenge: | Existing methods for controlling sentence difficulty have not taken into account sentence-level difficulties. |
| Approach: | They propose a method for controlling the difficulty of a sentence based on deep reinforcement learning. |
| Outcome: | The proposed method generates sentences of appropriate difficulty for the target audience through reinforcement learning. |
Deep Reinforcement Learning for NLP (P18-5)
Copied to clipboard
| Challenge: | Many natural language processing tasks can be formulated as deep reinforcement learning (DRL) problems. |
| Approach: | This tutorial provides an introduction to the foundations of deep reinforcement learning . it describes recent advances in designing deep reinforcement for NLP . |
| Outcome: | This tutorial provides an introduction to the foundations of deep reinforcement learning and some practical solutions for NLP tasks. |
Paraphrase Generation with Deep Reinforcement Learning (D18-1)
Copied to clipboard
| Challenge: | Paraphrase generation is an important but challenging task in natural language processing . traditional symbolic approaches to paraphrase generation include rule-based methods, thesaurus-based approaches and statistical machine translation (SMT) |
| Approach: | They propose a deep reinforcement learning approach to automatic paraphrase generation . they propose supervised learning and reinforcement learning for evaluators . |
| Outcome: | The proposed framework outperforms state-of-the-art methods in paraphrase generation on two datasets. |