Papers with MAB
One Cannot Stand for Everyone! Leveraging Multiple User Simulators to train Task-oriented Dialogue Systems (2023.acl-long)
Copied to clipboard
| Challenge: | Recent studies have found that Task-oriented Dialogue systems can be more suitable for human users. |
| Approach: | They propose a framework to optimize ToD systems by leveraging Multiple User SimulaTors. |
| Outcome: | The proposed framework improves performance on multiWOZ with human evaluations and automatic evaluations. |
Chunks as Arms: Multi-Armed Bandit-Guided Sampling for Long-Context LLM Preference Optimization (2026.acl-long)
Copied to clipboard
Shaohua Duan, Pengcheng Huang, Xinze Li, Zhenghao Liu, Xiaoyuan Yi, Yukun Yan, Shuo Wang, Yu Gu, Ge Yu, Maosong Sun
| Challenge: | Recent studies have explored fine-tuning Large Language Models with synthetic data to enhance their long-context capabilities. |
| Approach: | They propose a framework that leverages a Multi-Armed Bandit rollout strategy to identify the most informative chunks from the given long context for sampling high-quality and diverse responses. |
| Outcome: | The proposed framework achieves 4% improvement on long-context reasoning benchmarks on Llama and Qwen. |
AutoRAG-HP: Automatic Online Hyper-Parameter Tuning for Retrieval-Augmented Generation (2024.findings-emnlp)
Copied to clipboard
Jia Fu, Xiaoting Qin, Fangkai Yang, Lu Wang, Jue Zhang, Qingwei Lin, Yubo Chen, Dongmei Zhang, Saravan Rajmohan, Qi Zhang
| Challenge: | Recent advances in Large Language Models have transformed ML/AI development . a reevaluation of AutoML principles for Retrieval-Augmented Generation (RAG) systems is needed. |
| Approach: | They propose a framework for hyper-parameter tuning and a hierarchical MAB method for efficient exploration of large search spaces. |
| Outcome: | The proposed framework outperforms baseline methods in more challenging optimization scenarios. |
LLMs are Biased Teachers: Evaluating LLM Bias in Personalized Education (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies have shown that relying on LLMs as information providers may hurt student learning. |
| Approach: | They introduce and apply two bias score metrics to evaluate LLMs for bias in the personalized educational setting, specifically on the models’ roles as “teachers.” |
| Outcome: | The proposed models harm student learning by perpetuating harmful stereotypes and reversing them. |
Large Language Model-Enhanced Multi-Armed Bandits (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been used to sequential decision-making tasks like multi-armed bandits where an LLM is tasked with selecting arms in each iteration is often suboptimal. |
| Approach: | They propose to combine MAB and LLMs to leverage the in-context learning capability of LLM for reward prediction. |
| Outcome: | The proposed approach outperforms LLM-based direct arm selection on synthetic tasks where only preference feedback between arm pairs is available. |