Natural Language-based State Representation in Deep Reinforcement Learning (2024.findings-naacl)
Copied to clipboard
| Challenge: | a new method for learning policies from images is proposed to reduce image-based observations' complexity and improve interpretability. |
| Approach: | They propose a method that compresses images into a natural language form for state representation. |
| Outcome: | The proposed method allows better interpretability and leverages processing capabilities of large-language models. |
Similar Papers
Reader: Model-based language-instructed reinforcement learning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing models of RL are limited and need to be re-trained for every new problem. |
| Approach: | They propose a model-based reinforcement learning approach to tackle the environment Read To Fight Monsters, a grounded policy learning problem. |
| Outcome: | The proposed approach performs better than existing model-free SOTA agents in the read to fight monsters environment and is more sample efficient than existing models. |
Deep Reinforcement Learning for NLP (P18-5)
Copied to clipboard
| Challenge: | Many natural language processing tasks can be formulated as deep reinforcement learning (DRL) problems. |
| Approach: | This tutorial provides an introduction to the foundations of deep reinforcement learning . it describes recent advances in designing deep reinforcement for NLP . |
| Outcome: | This tutorial provides an introduction to the foundations of deep reinforcement learning and some practical solutions for NLP tasks. |
Learning Natural Language Generation with Truncated Reinforcement Learning (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to train conditional languagemodels without supervised learning fail to scale to large action spaces, thus allowing to train a language agent by only interacting with its environment without any task-specific prior knowledge. |
| Approach: | They propose an original approach to train conditional languagemodels without supervised learning by only using reinforcement learning. |
| Outcome: | The proposed approach avoids the dependency to labelled datasets and reduces pretrained policy flaws such as language or exposure biases. |
Improving the Language Understanding Capabilities of Large Language Models Using Reinforcement Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Instruction-fine-tuned large language models (LLMs) under 14B parameters underperform on NLU tasks . we explore a framework to improve the NLU capabilities of LLMs . |
| Approach: | They propose to use Proximal Policy Optimization to improve NLU capabilities . they frame NLU as a reinforcement learning environment and optimize for reward signals . |
| Outcome: | The proposed framework outperforms supervised fine-tuning on GLUE and superGLUE tasks. |
Improving Reinforcement Learning Based Image Captioning with Natural Language Prior (D18-1)
Copied to clipboard
| Challenge: | Recent research shows that Reinforcement Learning (RL) approaches suffer from the exposure bias problem. |
| Approach: | They propose a Reinforcement Learning (RL) based training framework that constrains the action space using an n-gram language prior. |
| Outcome: | The proposed model is more human readable and graceful. |
Adaptive Reinforcement Tuning Language Models as Hard Data Generators for Sentence Representation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods use contrastive learning (CL) to learn effective sentence representations, but require extensive human annotation. |
| Approach: | They propose a reinforcement learning approach for fine-tuning small-parameter LLMs to generate high-quality hard contrastive data without human feedback. |
| Outcome: | The proposed method achieves state-of-the-art on seven semantic text similarity tasks. |
Learning with Latent Language (N18-1)
Copied to clipboard
| Challenge: | Using the space of natural language strings as a parameter space is an effective way to capture natural task structure. |
| Approach: | They propose to use natural language as a parameter space for few-shot learning problems including classification, transduction and policy search. |
| Outcome: | The proposed model outperforms models with a linguistic parameterization on image classification, text editing, and reinforcement learning. |
A Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and Solutions (2026.acl-long)
Copied to clipboard
Zhiyin Yu, Yuchen Mou, Juncheng Yan, Junyu Luo, Chunchun Chen, Xing Wei, Yunhui Liu, Hongru Sun, Yuxing Zhang, Jun Xu, Yatao Bian, Ming Zhang, Wei Ye, Tieke He, Jie Yang, Guanjie Zheng, Zhonghai Wu, Bo Zhang, Lei Bai, Xiao Luo
| Challenge: | Existing research on reinforcement learning for LLMs under data scarcity has not been unified. |
| Approach: | They propose a top-up hierarchical framework built around three complementary perspectives: data-centric, training-centric and framework-centric. |
| Outcome: | The proposed framework provides a clear conceptual foundation for understanding the design space of data-efficient RL for large language models and to guide researchers working in this emerging area. |
Large Language Models Can Learn Representation in Natural Language (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are unable to complete complex tasks involving multiple entities, such as tool APIs. |
| Approach: | They propose a method which uses natural language representations to refine entity descriptions for improved retrieval and LLM utilization. |
| Outcome: | The proposed method improves GPT-4's performance on classification tasks and API call tasks. |
Deep Bayesian Natural Language Processing (P19-4)
Copied to clipboard
| Challenge: | Introduction to deep Bayesian learning for natural language addresses the fundamentals of statistical models and neural networks. |
| Approach: | This tutorial addresses the advances in deep Bayesian learning for natural language . it focuses on advanced Bayessian models and deep models . authors present case studies and domain applications to tackle different issues . |
| Outcome: | This tutorial focuses on advanced Bayesian models and deep models for natural language . case studies and domain applications are presented to tackle different issues in deep Bayessian processing, learning and understanding. |