Generalization in Text-based Games via Hierarchical Reinforcement Learning (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Reinforcement Learning (RL) based agents are promising for text-based games, but their generalization remains a challenge. |
| Approach: | They propose a hierarchical framework for reinforcement learning based on knowledge graphs . they propose to decompose the game into subtasks and execute a sub-policy in the low level to conduct goal-conditioned reinforcement learning. |
| Outcome: | The proposed framework enjoys favorable generalizability on a set of difficulty levels and is able to handle complex training tasks. |
Similar Papers
Playing Text-Adventure Games with Graph-Based Deep Reinforcement Learning (N19-1)
Copied to clipboard
| Challenge: | Text adventure games provide a platform for exploring reinforcement learning in combinatorial action space, such as natural language. |
| Approach: | They propose a deep reinforcement learning architecture that represents the game state as a knowledge graph which is learned during exploration. |
| Outcome: | The proposed architecture can learn a control policy faster than baseline alternatives. |
Policy-based Reinforcement Learning for Generalisation in Interactive Text-based Environments (2023.eacl-main)
Copied to clipboard
| Challenge: | Text-based environments allow RL agents to learn to converse and perform interactive tasks through natural language. |
| Approach: | They propose to switch from a value-based update method to a policy-based one within text-based environments and evaluate it on Coin Collector and Question Answering with interactive text (QAit). |
| Outcome: | The proposed policy-based agent is more generalised than value-based methods in two text-based environments designed to test zero-shot performance. |
KEHRL: Learning Knowledge-Enhanced Language Representations with Hierarchical Reinforcement Learning (2024.lrec-main)
Copied to clipboard
| Challenge: | General pre-trained language models (PLMs) leverage relation triples from knowledge graphs (KGs) and integrate external data sources into language models via self-supervised learning. |
| Approach: | They propose to learn Knowledge-Enhanced language representations with Hierarchical Reinforcement Learning (KEHRL) to detect positions for knowledge injection and integrate external knowledge into the model to avoid injecting inaccurate or irrelevant knowledge. |
| Outcome: | The proposed model can detect essential positions in texts for knowledge injection and integrate external knowledge into the model to avoid injecting inaccurate or irrelevant knowledge. |
A Survey of Text Games for Reinforcement Learning Informed by Natural Language (2022.tacl-1)
Copied to clipboard
| Challenge: | Interactive Fiction Games (text games) are a problem type that require natural language to solve complex tasks. |
| Approach: | They propose to use interactive fiction games as a testing environment to test the new Reinforcement Learning solutions using natural language. |
| Outcome: | The proposed solutions are based on the proposed interactive fiction games and the generated environments. |
Controllable Text Simplification with Deep Reinforcement Learning (2022.aacl-short)
Copied to clipboard
| Challenge: | Existing methods for controlling sentence difficulty have not taken into account sentence-level difficulties. |
| Approach: | They propose a method for controlling the difficulty of a sentence based on deep reinforcement learning. |
| Outcome: | The proposed method generates sentences of appropriate difficulty for the target audience through reinforcement learning. |
Perceiving the World: Question-guided Reinforcement Learning for Text-based Games (2022.acl-long)
Copied to clipboard
| Challenge: | Text-based games provide an interactive way to study natural language processing. |
| Approach: | They propose a two-phase training framework to decouple language learning from reinforcement learning and improve the sample efficiency. |
| Outcome: | The proposed method significantly improves performance and sample efficiency against compound error and limited pre-training data. |
STARLING: Self-supervised Training of Text-based Reinforcement Learning Agent with Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing environments for text-based RL are domain-specific or time-consuming to generate and do not train the agents to master a specific set of skills. |
| Approach: | They propose an interactive environment for self-supervised RL that bootstraps the text-based RL agents with automatically generated games to boost their generalization capabilities. |
| Outcome: | The proposed environment bootstraps the agents with automatically generated games to boost their generalization capabilities to reach a goal of the target environment. |
Reinforcement Learning with Token-level Feedback for Controllable Text Generation (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for controllable text generation are guided by coarse-grained feedback, which may lead to suboptimal performance owing to semantic twists or progressions within sentences. |
| Approach: | They propose a reinforcement learning algorithm which formulates TOken-LEvel rewards for controllable text generation and employs a "first-quantize-then-noise" paradigm to enhance the robustness of the RL algorithm. |
| Outcome: | The proposed algorithm can achieve superior performance on single-attribute and multi-attract control tasks. |
Transfer in Deep Reinforcement Learning Using Knowledge Graphs (D19-53)
Copied to clipboard
| Challenge: | Text adventure games provide a stepping stone toward grounding action in language . prior work demonstrated that using a knowledge graph as a state representation facilitates faster control policy learning. |
| Approach: | They propose to use knowledge graphs as a representation for domain knowledge transfer for training text-adventure playing reinforcement learning agents. |
| Outcome: | The proposed methods let us learn a higher-quality control policy faster in text adventure games. |
Why Does Reinforcement Learning Generalize? A Feature-Level Mechanistic Study of Post-Training in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Reinforcement learning (RL)-based post-training often improves the reasoning performance of large language models beyond the training domain, while supervised fine-tuning (SFT) frequently leads to general capabilities forgetting. |
| Approach: | They propose a feature-level mechanistic analysis methodology to probe RL generalization using a controlled experimental setup. |
| Outcome: | The proposed method identifies a compact, task-agnostic set of features that directly mediate generalization across diverse tasks. |