Progra: Progress-Aware Reinforcement Learning for Multi-Turn Function Calling (2026.findings-acl)
Copied to clipboard
Huacan Chai, Zijie Cao, Maolin Ran, Yingxuan Yang, Jianghao Lin, Xin Peng, Hairui Wang, Renjie Ding, Ziyu Wan, Muning Wen, Weiwen Liu, Weinan Zhang, Fei Huang, Ying Wen
| Challenge: | Existing methods for multi-turn function calling are limited by redundancy and lack explicit integration of progress awareness into training. |
| Approach: | They propose a framework that explicitly integrates progress awareness into LLM training for multi-turn function calling. |
| Outcome: | Empirical results show that Progra outperforms existing methods on two public benchmarks. |
Similar Papers
Turn-PPO: Turn-Level Advantage Estimation with PPO for Improved Multi-Turn RL in Agentic LLMs (2026.findings-eacl)
Copied to clipboard
| Challenge: | Reinforcement learning (RL) has re-emerged as a natural approach for training interactive LLM agents in real-world environments. |
| Approach: | They propose a variant that operates on a turn-level MDP formulation, instead of the commonly used token-level one. |
| Outcome: | The proposed method is more robust than the widely used GRPO algorithm and more efficient than token-level MDPs. |
STEP: Success-Rate-Aware Trajectory-Efficient Policy Optimization (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing GRPO-based methods allocate sampling uniformly across tasks regardless of difficulty, propagate misleading learning signals and incur high sample-collection costs. |
| Approach: | They propose a framework that allocates sampling based on per-task success rates and performs fine-grained step-level optimization. |
| Outcome: | The proposed method improves sample efficiency and training stability over existing GRPO variants and three ablation variants on OSWorld and AndroidWorld. |
Enhancing Efficiency and Exploration in Reinforcement Learning for LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches allocate an equal number of rollouts to all questions during the RL process, which is inefficient. |
| Approach: | They propose a mechanism for dynamically allocating rollout budgets based on the difficulty of the problems, enabling more efficient RL training. |
| Outcome: | The proposed model improves response precision while preserving exploratory ability to uncover potential correct pathways. |
Empowering Multi-Turn Tool-Integrated Agentic Reasoning with Group Turn Policy Optimization (2026.acl-long)
Copied to clipboard
Yifeng Ding, Hung Le, Songyang Han, Kangrui Ruan, Zhenghui Jin, Varun Kumar, Zijian Wang, Anoop Deoras
| Challenge: | Current reinforcement learning methods suffer from coarse-grained, trajectory-level rewards that provide insufficient learning signals for complex multi-turn interactions, leading to training stagnation. |
| Approach: | They propose a novel RL algorithm for training large language models for multi-turn tool-integrated reasoning (TIR) that incorporates three innovations: turn-level reward assignment that provides fine-grained feedback for individual turns, return-based advantage estimation where normalized discounted returns are calculated as advantages, and self-supervised reward shaping that exploits self-supervision signals from generated code to densify sparse binary outcome-based rewards. |
| Outcome: | The proposed algorithm outperforms GRPO by 3.0% across diverse math reasoning benchmarks and improves grepo by 3.9% on commonsense reasoning and program synthesis tasks. |
Improving the Language Understanding Capabilities of Large Language Models Using Reinforcement Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Instruction-fine-tuned large language models (LLMs) under 14B parameters underperform on NLU tasks . we explore a framework to improve the NLU capabilities of LLMs . |
| Approach: | They propose to use Proximal Policy Optimization to improve NLU capabilities . they frame NLU as a reinforcement learning environment and optimize for reward signals . |
| Outcome: | The proposed framework outperforms supervised fine-tuning on GLUE and superGLUE tasks. |
KARL: Reinforcement Learning for LLM Agents on Multi-Turn Knowledge-Intensive Agentic Tasks (2026.acl-long)
Copied to clipboard
Xueqiao Sun, Xiao Liu, Bowen Lv, Hanchen Zhang, Bohao Jing, Zehan Qi, Yifan Xu, Yuxiao Dong, Jie Tang
| Challenge: | Large Language Models have shown remarkable potential as autonomous agents, but their effectiveness in knowledge-intensive tasks remains limited by passive knowledge utilization. |
| Approach: | They propose a framework that enables LLM agents to dynamically explore structured knowledge sources through multi-turn interactions. |
| Outcome: | The proposed framework outperforms existing retrieval-augmented approaches on knowledge graph and database tasks while maximizing tool-use behaviors end-to-end. |
GenesisFunc: Multi-Agent Data Generation for Accurate and Generalizable Function-Calling (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) extend their capabilities through function-calling (FC) however, obtaining and annotating real function-called data is challenging, and synthetic data from existing pipelines suffers from unreliable APIs, limited tool scalability, insufficient diversity, and weak quality control. |
| Approach: | They propose a pipeline for generating FC training data using reliable tools and a multi-agent framework that supports a dialogue generation system that produces conversations spanning diverse scenarios. |
| Outcome: | The proposed pipeline outperforms open-source models in in-domain FC performance and out-of-domain generalization while reaching FC capabilities comparable to some of the latest API-based models. |
Proactive Guidance of Multi-Turn Conversation in Industrial Search (2025.acl-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have advanced multi-turn conversation systems, emphasizing the need for proactive guidance to enhance users’ interactions. |
| Approach: | They propose a goal-adaptive supervised fine-tuning framework that generates proactive guidance for users to click for the next turn of the conversation. |
| Outcome: | The proposed framework achieves 86.10% accuracy in offline evaluation (+23.95% over baseline) and 25.28% CTR in online deployment (149.06% relative improvement). |
WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learning (2025.emnlp-main)
Copied to clipboard
Zhepei Wei, Wenlin Yao, Yao Liu, Weizhi Zhang, Qin Lu, Liang Qiu, Changlong Yu, Puyang Xu, Chao Zhang, Bing Yin, Hyokun Yun, Lihong Li
| Challenge: | Existing work on reinforcement learning has focused on single-turn tasks such as solving math problems. |
| Approach: | They propose a framework that learns directly from online interactions by asynchronously generating diverse trajectories, guided by binary rewards depending on task success. |
| Outcome: | Experiments on the WebArena-Lite benchmark show that the framework outperforms state-of-the-art methods and strong proprietary models. |
Aligning Paralinguistic Understanding and Generation in Speech LLMs via Multi-Task Reinforcement Learning (2026.eacl-industry)
Copied to clipboard
Minseok Kim, Jingxiang Chen, Seong-Gyun Leem, Yin Huang, Rashi Rungta, Zhicheng Ouyang, Haibin Wu, Surya Teja Appini, Ankur Bansal, Yang Bai, Yue Liu, Florian Metze, Ahmed A Aly, Anuj Kumar, Ariya Rastrow, Zhaojiang Lin
| Challenge: | Using paralinguistic cues is challenging for speech large language models, authors say . limited training data, annotation difficulty, and models exploiting lexical shortcuts are challenges . a recent study shows that modeling paralinguistic reasoning with multitask RL improves paralinguistics understanding . |
| Approach: | They propose multi-task reinforcement learning with chain-of-thought prompting that elicits explicit affective reasoning. |
| Outcome: | The proposed model improves paralinguistics understanding over baselines and strong proprietary models by 8-12% on Expresso, IEMOCAP, and RAVDESS. |