Fire Burns, Sword Cuts: Commonsense Inductive Bias for Exploration in Text-based Games (2022.acl-short)
Copied to clipboard
| Challenge: | Existing RL agents are far away from solving text-based games due to their combinatorially large action spaces that hinders efficient exploration. |
| Approach: | They propose an exploration technique that injects external commonsense knowledge, via a pretrained language model, into the agent during training when the agent is the most uncertain about its next action. |
| Outcome: | The proposed method exhibits improvement on the collected game scores during the training in four out of nine games from Jericho. |
Similar Papers
EXPLORER: Exploration-guided Reasoning for Textual Reinforcement Learning (2024.eacl-long)
Copied to clipboard
Kinjal Basu, Keerthiram Murugesan, Subhajit Chaudhury, Murray Campbell, Kartik Talamadupula, Tim Klinger
| Challenge: | Text-based games (TBGs) combine natural language understanding with reasoning. |
| Approach: | They propose an exploration-guided reasoning agent for textual reinforcement learning that integrates natural language with reasoning. |
| Outcome: | The proposed agent outperforms baseline agents on TWG and TWC games. |
Keep CALM and Explore: Language Models for Action Generation in Text-based Games (2020.emnlp-main)
Copied to clipboard
| Challenge: | Text-based games present a unique challenge for autonomous agents to operate in natural language and handle enormous action spaces. |
| Approach: | They propose a Contextual Action Language Model (CALM) to generate a compact set of action candidates at each game state. |
| Outcome: | The proposed model achieves a 69% improvement in average game score on unsupervised games . the proposed model is competitive with or better than other models that have access to ground truth admissible actions on half of the games tested . |
Perceiving the World: Question-guided Reinforcement Learning for Text-based Games (2022.acl-long)
Copied to clipboard
| Challenge: | Text-based games provide an interactive way to study natural language processing. |
| Approach: | They propose a two-phase training framework to decouple language learning from reinforcement learning and improve the sample efficiency. |
| Outcome: | The proposed method significantly improves performance and sample efficiency against compound error and limited pre-training data. |
Efficient Text-based Reinforcement Learning by Jointly Leveraging State and Commonsense Graph Representations (2021.acl-short)
Copied to clipboard
Keerthiram Murugesan, Mattia Atzeni, Pavan Kapanipathi, Kartik Talamadupula, Mrinmaya Sachan, Murray Campbell
| Challenge: | Text-based games (TBGs) are useful benchmarks for evaluating progress in grounded language understanding and reinforcement learning (RL). |
| Approach: | They propose an agent that induces a graph representation of the game state and jointly grounds it with a commonsense knowledge from ConceptNet. |
| Outcome: | The proposed agent outperforms baseline agents in the proposed game . |
PLM-based World Models for Text-based Games (2022.emnlp-main)
Copied to clipboard
| Challenge: | a new study shows that pre-trained world models provide a strong base for world models . worldformer is a text-based game environment that can be used to learn world models in text-driven games. |
| Approach: | They propose to use pre-trained language models to build world models in text-based game environments. |
| Outcome: | The proposed model outperforms state-of-the-art model-free algorithms in Atari games while retaining sample efficiency. |
Playing Text-Adventure Games with Graph-Based Deep Reinforcement Learning (N19-1)
Copied to clipboard
| Challenge: | Text adventure games provide a platform for exploring reinforcement learning in combinatorial action space, such as natural language. |
| Approach: | They propose a deep reinforcement learning architecture that represents the game state as a knowledge graph which is learned during exploration. |
| Outcome: | The proposed architecture can learn a control policy faster than baseline alternatives. |
Learn What Is Possible, Then Choose What Is Best: Disentangling One-To-Many Relations in Language Through Text-based Games (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Language models pre-trained on large self-supervised corpora, followed by task-specific fine-tuning has become the dominant paradigm in NLP. |
| Approach: | They propose to train language models pre-trained on large self-supervised corpora, followed by task-specific fine-tuning on the target domain. |
| Outcome: | The proposed model improves on the previous state-of-the-art model on the Jericho Walkthroughs dataset by 49%. |
VEG: Verbal đťś–-greedy for Semantic Exploration in Multi-Turn RL Agents (2026.acl-industry)
Copied to clipboard
Yongchang Hao, Jie Hao, Yongsheng Mei, Ze Ye, Junyi Chai, Bin Guo, Benjamin Z. Yao, Chenlei Guo, Lili Mou
| Challenge: | Standard RL approaches suffer from reward sparsity and mode-seeking behavior . lack of diversity hinders exploration necessary for optimal learning . |
| Approach: | They propose a framework that leverages external feedback as a dynamic control variable to explicitly balance exploration and exploitation within the semantic space. |
| Outcome: | Experiments on Tau Bench and SearchQA show that the proposed framework outperforms standard RL baselines. |
Language Guided Exploration for RL Agents in Text Environments (2024.findings-naacl)
Copied to clipboard
| Challenge: | Real-world sequential decision making is characterized by sparse rewards and large decision spaces. |
| Approach: | They introduce a language-based framework that provides decision-level guidance to an RL agent. |
| Outcome: | The proposed framework outperforms vanilla RL agents on ScienceWorld in 2022. |
Emergent Communication Pretraining for Few-Shot Machine Translation (2020.coling-main)
Copied to clipboard
| Challenge: | state-of-the-art models that rely on multilingual pretrained encoders achieve sample efficiency in downstream applications, but lack abundant amounts of unlabelled text. |
| Approach: | They propose a method to pretrain neural networks via emergent communication from referential games by grounding communication on images as a crude approximation of real-world environments. |
| Outcome: | The proposed method significantly improves machine translation in few-shot settings and provides an evaluation protocol to probe the properties of emergent languages ex vitro. |