Papers with Q-learning
Q-learning with Language Model for Edit-based Unsupervised Summarization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Unsupervised text summarization methods are promising, but their performance is still behind that of state-of-the-art supervised methods. |
| Approach: | They propose a method based on Q-learning with an edit-based summarization that uses an Editorial Agent and Language Model converter to predict edit actions. |
| Outcome: | The proposed method delivers competitive performance even with zero paired data, while requiring no validation set. |
Convert Language Model into a Value-based Strategic Planner (2025.acl-industry)
Copied to clipboard
| Challenge: | Emotional support conversation (ESC) aims to alleviate the emotional distress of individuals through effective conversations. |
| Approach: | They propose a framework that bootstraps the planning during ESC and determines the optimal strategy based on long-term returns. |
| Outcome: | The proposed framework outperforms baseline models on ESC datasets and can be used to guide the LLM to response. |
Reinforced Co-Training (N18-1)
Copied to clipboard
| Challenge: | Existing co-training methods ignore sampling bias and fail to explore the data space. |
| Approach: | They propose a method to select high-quality unlabeled samples to better co-train on by learning a selection policy with a small labeled dataset. |
| Outcome: | The proposed method can obtain more accurate text classification results on clickbait detection and generic text classification tasks. |
Bootstrapped Q-learning with Context Relevant Observation Pruning to Generalize in Text-based Games (2020.emnlp-main)
Copied to clipboard
Subhajit Chaudhury, Daiki Kimura, Kartik Talamadupula, Michiaki Tatsubori, Asim Munawar, Ryuki Tachibana
| Challenge: | Reinforcement Learning methods for text-based games fail to generalize on unseen games, especially in small data regimes. |
| Approach: | They propose a Context Relevant Episodic State Truncation method for irrelevant token removal in observation text for improved generalization. |
| Outcome: | The proposed method shows that it can generalize on unseen games using 10x-20x fewer training games compared to previous state-of-the-art methods despite requiring fewer number of training episodes. |
Efficient (Soft) Q-Learning for Text Generation with Limited Good Data (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Maximum likelihood estimation (MLE) is the predominant method for training text generation models. |
| Approach: | They propose a new RL formulation for text generation from the soft Q-learning perspective using path consistency learning to combine the best of on-/off-policy updates and learn effectively from sparse reward. |
| Outcome: | The proposed approach outperforms MLE and previous RL methods in a wide range of tasks. |