LLaMA-Rider: Spurring Large Language Models to Explore the Open World (2024.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies have used Large Language Models to help decision-making and planning in environments, but their capacity to acquire environmental knowledge and adapt in an open world remains uncertain. |
| Approach: | They propose an approach to spur LLMs to explore the open world, gather experiences, and learn to improve their task-solving capabilities by using a feedback-revision mechanism. |
| Outcome: | The proposed model enhances the efficiency of the LLM in exploring the open world and improves its ability to accomplish more tasks through fine-tuning with merely 1.3k instances of collected data, showing minimal training costs compared to baseline using reinforcement learning. |
Similar Papers
Improving the Language Understanding Capabilities of Large Language Models Using Reinforcement Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Instruction-fine-tuned large language models (LLMs) under 14B parameters underperform on NLU tasks . we explore a framework to improve the NLU capabilities of LLMs . |
| Approach: | They propose to use Proximal Policy Optimization to improve NLU capabilities . they frame NLU as a reinforcement learning environment and optimize for reward signals . |
| Outcome: | The proposed framework outperforms supervised fine-tuning on GLUE and superGLUE tasks. |
Language Adaptation of Large Language Models: An Empirical Study on LLaMA2 (2025.coling-main)
Copied to clipboard
| Challenge: | Popularity of Large Language Models (LLMs) has seen a skyrocketing increase in recent years. |
| Approach: | They present a systematic review of the language adaptation process for Large Language Models including vocabulary expansion, continued pre-training, and instruction fine-tuning. |
| Outcome: | The proposed model is based on empirical studies conducted on LLaMA2 and discussions on various settings affecting the model's capabilities. |
Feedback-Driven Tool-Use Improvements in Large Language Models via Automated Build Environments (2026.findings-acl)
Copied to clipboard
Junjie Ye, Changhao Jiang, Zhengyin Du, Yufei Xu, Xuesong Yao, Zhiheng Xi, Xiaoran Fan, Qi Zhang, Tao Gui, Xuanjing Huang, Jiecao Chen
| Challenge: | Currently, there are no efficient reinforcement learning (RL) frameworks specifically designed for tool use. |
| Approach: | They propose an automated environment construction pipeline that incorporates scenario decomposition, document generation, function integration, complexity scaling, and localized deployment to enable high-quality training environments without external tools. |
| Outcome: | The proposed framework significantly improves the models’ tool-use performance without degrading their general capabilities. |
Language Guided Exploration for RL Agents in Text Environments (2024.findings-naacl)
Copied to clipboard
| Challenge: | Real-world sequential decision making is characterized by sparse rewards and large decision spaces. |
| Approach: | They introduce a language-based framework that provides decision-level guidance to an RL agent. |
| Outcome: | The proposed framework outperforms vanilla RL agents on ScienceWorld in 2022. |
Enhancing Efficiency and Exploration in Reinforcement Learning for LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches allocate an equal number of rollouts to all questions during the RL process, which is inefficient. |
| Approach: | They propose a mechanism for dynamically allocating rollout budgets based on the difficulty of the problems, enabling more efficient RL training. |
| Outcome: | The proposed model improves response precision while preserving exploratory ability to uncover potential correct pathways. |
From Tarzan to Tolkien: Controlling the Language Proficiency Level of LLMs for Content Generation (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) often output text at a native level of speech, making them difficult to use for contexts where end-users are not fully proficient. |
| Approach: | They propose a framework to control the difficulty level of text generated by Large Language Models for contexts where end-users are not fully proficient. |
| Outcome: | The proposed framework surpasses GPT-4 and other models at fraction of the cost. |
AdaRefiner: Refining Decisions of Language Models with Adaptive Feedback (2024.findings-naacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated significant success across various domains, but their application in complex decision-making tasks often necessitates intricate prompt engineering or fine-tuning. |
| Approach: | They propose a lightweight Adapter Language Model (LM) which automatically refines task comprehension based on feedback from RL agents. |
| Outcome: | The proposed framework enhances synergy between LLMs and RL feedback while maintaining generalization abilities and enhancing decision-making capabilities in downstream tasks. |
Can Small Language Models Help Large Language Models Reason Better?: LM-Guided Chain-of-Thought (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing frameworks for guiding a language model in reasoning tasks are limited by their tendency to generate low-quality rationales that are repetitive and vacuous. |
| Approach: | They propose a framework that leverages a lightweight language model for guiding a black-box large LM in reasoning tasks. |
| Outcome: | The proposed framework outperforms baselines in answer prediction accuracy. |
LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models (2024.acl-demos)
Copied to clipboard
| Challenge: | Efficient fine-tuning of large language models requires non-trivial efforts to implement these methods on different models. |
| Approach: | They propose a framework that democratizes the fine-tuning of large language models by integrating a suite of efficient training methods into one framework. |
| Outcome: | The proposed framework is able to scale to 100+ LLMs without coding and receives over 25,000 stars and 3,000 forks. |
VEEF-Multi-LLM: Effective Vocabulary Expansion and Parameter Efficient Finetuning Towards Multilingual Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have a significant disadvantage for low-resource languages . VEEF-Multi-LLM-8B excels in multilingual instruction-following tasks . |
| Approach: | They propose a low-resource multilingual large language model that expands the vocabulary for multilingual support. |
| Outcome: | The proposed model outperforms existing models in multilingual instruction-following tasks, but lags behind English-centric models in some tasks. |