Policy-based Reinforcement Learning for Generalisation in Interactive Text-based Environments (2023.eacl-main)
Copied to clipboard
| Challenge: | Text-based environments allow RL agents to learn to converse and perform interactive tasks through natural language. |
| Approach: | They propose to switch from a value-based update method to a policy-based one within text-based environments and evaluate it on Coin Collector and Question Answering with interactive text (QAit). |
| Outcome: | The proposed policy-based agent is more generalised than value-based methods in two text-based environments designed to test zero-shot performance. |
Similar Papers
STARLING: Self-supervised Training of Text-based Reinforcement Learning Agent with Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing environments for text-based RL are domain-specific or time-consuming to generate and do not train the agents to master a specific set of skills. |
| Approach: | They propose an interactive environment for self-supervised RL that bootstraps the text-based RL agents with automatically generated games to boost their generalization capabilities. |
| Outcome: | The proposed environment bootstraps the agents with automatically generated games to boost their generalization capabilities to reach a goal of the target environment. |
Neuro-Symbolic Approaches for Text-Based Policy Learning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Text-based games are important testbeds for reinforcement learning in the natural language domain. |
| Approach: | They propose a method that learns interpretable action policy rules from symbolic abstractions of textual observations for improved generalization. |
| Outcome: | The proposed method outperforms existing methods in RL using 5-10x fewer training games. |
On the Effects of Fine-tuning Language Models for Text-Based Reinforcement Learning (2025.coling-main)
Copied to clipboard
| Challenge: | Text-based reinforcement learning is a form of interactive fiction where players manipulate the environment using text and admissible actions in natural language. |
| Approach: | They show that rich semantic understanding leads to efficient training of text-based RL agents . they also show that semantic degeneration occurs when LMs are inappropriately fine-tuned . |
| Outcome: | The results suggest that semantic understanding is not important for the task . they also show that fine-tuning language models can degenerate the agent's performance . |
Language Guided Exploration for RL Agents in Text Environments (2024.findings-naacl)
Copied to clipboard
| Challenge: | Real-world sequential decision making is characterized by sparse rewards and large decision spaces. |
| Approach: | They introduce a language-based framework that provides decision-level guidance to an RL agent. |
| Outcome: | The proposed framework outperforms vanilla RL agents on ScienceWorld in 2022. |
Perceiving the World: Question-guided Reinforcement Learning for Text-based Games (2022.acl-long)
Copied to clipboard
| Challenge: | Text-based games provide an interactive way to study natural language processing. |
| Approach: | They propose a two-phase training framework to decouple language learning from reinforcement learning and improve the sample efficiency. |
| Outcome: | The proposed method significantly improves performance and sample efficiency against compound error and limited pre-training data. |
Generalization in Text-based Games via Hierarchical Reinforcement Learning (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Reinforcement Learning (RL) based agents are promising for text-based games, but their generalization remains a challenge. |
| Approach: | They propose a hierarchical framework for reinforcement learning based on knowledge graphs . they propose to decompose the game into subtasks and execute a sub-policy in the low level to conduct goal-conditioned reinforcement learning. |
| Outcome: | The proposed framework enjoys favorable generalizability on a set of difficulty levels and is able to handle complex training tasks. |
Reader: Model-based language-instructed reinforcement learning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing models of RL are limited and need to be re-trained for every new problem. |
| Approach: | They propose a model-based reinforcement learning approach to tackle the environment Read To Fight Monsters, a grounded policy learning problem. |
| Outcome: | The proposed approach performs better than existing model-free SOTA agents in the read to fight monsters environment and is more sample efficient than existing models. |
Language-based General Action Template for Reinforcement Learning Agents (2021.findings-acl)
Copied to clipboard
| Challenge: | Prior knowledge is important in decision-making, and humans preserve it in the form of natural language (NL). |
| Approach: | They propose an environmentagnostic action framework that incorporates prior knowledge into decision-making . they propose to use general semantic schemes to facilitate agent in finding plausible actions . |
| Outcome: | The proposed agent performs better than agents that rely on gamespecific actions. |
Self-imitation Learning for Action Generation in Text-based Games (2023.eacl-main)
Copied to clipboard
| Challenge: | Text-based games are situated systems where the game agents observe textual descriptions, and generate textual commands to interact with the environment. |
| Approach: | They propose a confidence-based self-imitation model to generate action candidates for the RL agent by exploiting past valuable trajectories to adapt a pre-trained language model towards a target game. |
| Outcome: | The proposed model performs well in multiple challenging games. |
From Word to World: Can Large Language Models be Implicit Text-based World Models? (2026.acl-long)
Copied to clipboard
Yixia Li, Hongru Wang, Jiahao Qiu, Zhenfei Yin, Dongdong Zhang, Cheng Qian, Zeping Li, Xiaoteng Ma, Guanhua Chen, Heng Ji
| Challenge: | Agentic learning increasingly hinges on interaction, yet real-world experience is expensive, limited, and often irreversible at inference time. |
| Approach: | They propose a framework that reframes language modeling as next-state prediction under interaction. |
| Outcome: | The proposed framework evaluates world models in text-based environments . it shows that sufficiently trained models capture coherent environment dynamics . |