Beyond Numeric Rewards: In-Context Dueling Bandits with LLM Agents (2025.findings-acl)
Copied to clipboard
| Challenge: | In-Context Reinforcement Learning (ICRL) is a frontier paradigm for RL problems . authors find that LLMs can generalize cross-domain to perform ICRL on a stateless preference-based RL problem. |
| Approach: | They propose an agentic-flow framework that integrates off-the-shelf DB algorithm support with LLM agents through fine-grained adaptive interplay. |
| Outcome: | The proposed framework can generalize cross-domain to perform ICRL on a stateless preference-based RL problem. |
Similar Papers
Large Language Model-Enhanced Multi-Armed Bandits (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been used to sequential decision-making tasks like multi-armed bandits where an LLM is tasked with selecting arms in each iteration is often suboptimal. |
| Approach: | They propose to combine MAB and LLMs to leverage the in-context learning capability of LLM for reward prediction. |
| Outcome: | The proposed approach outperforms LLM-based direct arm selection on synthetic tasks where only preference feedback between arm pairs is available. |
Jump Starting Bandits with LLM-Generated Prior Knowledge (2024.emnlp-main)
Copied to clipboard
| Challenge: | Contextual multi-armed bandits generate personalized recommendations based on user-specific contexts. |
| Approach: | They propose an initialization algorithm for contextual bandits by prompting LLMs to produce a pre-training dataset of approximate human preferences for the bandit. |
| Outcome: | The proposed approach significantly reduces online learning regret and data-gathering costs for training such models. |
Enough Coin Flips Can Make LLMs Act Bayesian (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models exhibit the ability to generalize given few-shot examples in their input prompt, an emergent capability known as in-context learning. |
| Approach: | They investigate whether large language models use in-context learning to generalize given few-shot examples in their input prompt. |
| Outcome: | The proposed model can generalize given few-shot examples in their input prompt, an emergent capability known as in-context learning. |
RLAE: Reinforcement Learning-Assisted Ensemble for LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing ensemble methods for ensembling large language models rely on fixed weighting strategies that fail to adapt to dynamic, context-dependent characteristics of LLMs. |
| Approach: | They propose a framework that reformulates LLM ensemble through a Markov Decision Process. |
| Outcome: | The proposed framework outperforms existing methods by 3.3% on a diverse set of tasks while achieving lower time latency. |
Chunks as Arms: Multi-Armed Bandit-Guided Sampling for Long-Context LLM Preference Optimization (2026.acl-long)
Copied to clipboard
Shaohua Duan, Pengcheng Huang, Xinze Li, Zhenghao Liu, Xiaoyuan Yi, Yukun Yan, Shuo Wang, Yu Gu, Ge Yu, Maosong Sun
| Challenge: | Recent studies have explored fine-tuning Large Language Models with synthetic data to enhance their long-context capabilities. |
| Approach: | They propose a framework that leverages a Multi-Armed Bandit rollout strategy to identify the most informative chunks from the given long context for sampling high-quality and diverse responses. |
| Outcome: | The proposed framework achieves 4% improvement on long-context reasoning benchmarks on Llama and Qwen. |
RIVAL: Reinforcement Learning with Iterative and Adversarial Optimization for Machine Translation (2025.findings-emnlp)
Copied to clipboard
Tianjiao Li, Mengran Yu, Chenyu Shi, Yanjun Zhao, Xiaojing Liu, Qi Zhang, Xuanjing Huang, Qiang Zhang, Jiayin Wang
| Challenge: | Using reinforcement learning from human feedback, large language models perform poorly when applied to colloquial subtitle translation tasks. |
| Approach: | They propose an adversarial training framework that iteratively updates the offline reward model and the online LLM to improve training outcomes. |
| Outcome: | The proposed training framework significantly improves upon translation baselines. |
LLMs for Bayesian Optimization in Scientific Domains: Are We There Yet? (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models have been proposed as general-purpose agents for experimental design . eval: LLMs show no sensitivity to experimental feedback. |
| Approach: | They propose a method that combines LLM prior knowledge with nearest-neighbor sampling to guide the design of experiments. |
| Outcome: | The proposed method outperforms classical methods in the design of experiments. |
Neuro-Symbolic Agentic Reinforcement Learning for Long-Term Original Character Companionship and Interaction (2026.acl-short)
Copied to clipboard
| Challenge: | Existing LLM-based agents that are optimized by prompting or supervised fine-tuning exhibit a generalization gap in long-horizon, socially rich interactions. |
| Approach: | They propose a framework that formalizes OC companion agents’ interactions as a POMDP and decomposes the agent into three sub-policies optimized via closed-loop RL from AI feedback with verifiable rewards in a graph-constrained action space. |
| Outcome: | The proposed framework formalizes OC companion agents’ interactions as a POMDP and decomposes the agent into three sub-policies (Router, Memory, and Persona) with verifiable rewards in a graph-constrained action space. |
ICL-Bandit: Relevance Labeling in Advertisement Recommendation Systems via LLM (2025.findings-emnlp)
Copied to clipboard
Lu Wang, Chiming Duan, Pu Zhao, Fangkai Yang, Yong Shi, Xuefeng Luo, Bingjing Xu, Weiwei Deng, Qingwei Lin, Dongmei Zhang
| Challenge: | In-context learning (ICL) is a common practice to enhance LLM performance on domain-specific tasks. |
| Approach: | They propose a method that leverages large language models to enhance query-ad relevance labeling . they identify and provide superior demonstrations for ICL, thereby improving labeling performance . |
| Outcome: | The proposed method improves query-ad relevance labeling performance by providing demonstrations. |
Dynamic Uncertainty Ranking: Enhancing Retrieval-Augmented In-Context Learning for Long-Tail Knowledge in LLMs (2025.naacl-long)
Copied to clipboard
| Challenge: | Prior work has shown that in-context learning (ICL) with retriever augmentation can help LLMs better capture long-tail knowledge, reducing their reliance on pre-trained data. |
| Approach: | They propose a reinforcement learning-based dynamic uncertainty ranking method that accounts for the varying impact of each retrieved sample on LLM predictions. |
| Outcome: | The proposed method outperforms baseline models on question-answering datasets by 2.76% and 5.96% on long-tail questions that elude zero-shot inference. |