| Challenge: | Multi-turn interactions between large language models and users naturally include implicit feedback signals. |
| Approach: | They propose a method to learn from feedback signals in past interactions without annotations . they use a multimodal LLM to solve a reasoning task with a combinatorial solution space . |
| Outcome: | The proposed method improves task completion rate from 31% to 82% without annotations. |
Similar Papers
Aligning LLMs with Individual Preferences via Interaction (2025.coling-main)
Copied to clipboard
| Challenge: | Existing studies on LLMs alignment focus on generalizing their behavior to generalized values such as helpfulness, harmlessness, and honesty. |
| Approach: | They train large language models to "interact to align" to implicitly infer user preferences . they use a multi-turn preference dataset to generate a personalized alignment . |
| Outcome: | The proposed method enables dynamic, personalized alignment via interaction with a multi-turn preference dataset. |
MetaReflection: Learning Instructions for Language Agents using Past Reflections (2024.emnlp-main)
Copied to clipboard
Priyanshu Gupta, Shashank Kirtania, Ananya Singha, Sumit Gulwani, Arjun Radhakrishna, Gustavo Soares, Sherry Shi
| Challenge: | Large Language Models (LLMs) have gained popularity due to their ability to generate human-like text and solve complex tasks. |
| Approach: | They propose an offline reinforcement learning technique that augments a semantic memory based on experiential learnings from past trials. |
| Outcome: | The proposed technique boosts Language agents’ performance by 4 % to 16.82 % over the raw GPT-4 baseline and performs on par with existing state-of-the-art prompt optimization techniques while requiring fewer LLM calls. |
WildFeedback: Aligning LLMs With In-situ User Interactions And Feedback (2026.acl-long)
Copied to clipboard
Taiwei Shi, Zhuoer Wang, Longqi Yang, Ying-Chun Lin, Zexue He, Mengting Wan, Pei Zhou, Sujay Kumar Jauhar, Sihao Chen, Shan Xia, Hongfei Zhang, Jieyu Zhao, Xiaofeng Xu, Xia Song, Jennifer Neville
| Challenge: | Traditional alignment methods rely on human annotations and are subjective and misalignment with real-world user preferences. |
| Approach: | They propose a framework that leverages in-situ user feedback during conversations with LLMs to create preference datasets automatically. |
| Outcome: | The proposed framework identifies and classifies user feedback to LLM responses between conversation turns and creates examples of preferred and dispreferred responses according to user preferences. |
User Feedback in Human-LLM Dialogues: A Lens to Understand Users But Noisy as a Learning Signal (2025.emnlp-main)
Copied to clipboard
| Challenge: | a recent study shows that asking for direct user feedback can be disruptive . we examine whether incorporating the contents of user feedback improves model performance . |
| Approach: | They analyze user feedback in the user-LLM conversation logs and harvest learning signals from it. |
| Outcome: | The proposed approach can lead to model degradation on two user-LM interaction datasets. |
Firm or Fickle? Evaluating Large Language Models Consistency in Sequential Interactions (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse tasks, but their deployment in high-stake domains requires consistent and coherent behavior across multiple rounds of user interaction. |
| Approach: | They propose a framework for evaluating and improving LLM response consistency, and introduce a benchmark dataset to evaluate LLM consistency. |
| Outcome: | The proposed framework improves response stability without sacrificing accuracy, and offers a practical path toward more dependable behavior in critical, real-world deployments. |
Exploring and Mitigating Shortcut Learning for Generative Large Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent large language models (LLMs) have incredible instruction-following capabilities while maintaining strong task completion ability. |
| Approach: | They propose a framework to encourage LLMs to Forget Spurious correlations and Learn from In-context information. |
| Outcome: | The proposed framework can mitigate shortcut learning by forging spurious correlations and learning from in-context information. |
Examining Inter-Consistency of Large Language Models Collaboration: An In-depth Analysis via Debate (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies focus on inconsistency issues within a single LLM, while we explore the inter-consistencies among multiple LLMs for collaboration. |
| Approach: | They propose a formal debate framework to examine whether LLMs can collaborate effectively to achieve a consensus for a shared goal. |
| Outcome: | The proposed framework enables LLMs to achieve consensus in three real-world debate scenarios with real-time scenarios aligned to the LLM's goals. |
What Did I Do Wrong? Quantifying LLMs’ Sensitivity and Consistency to Prompt Engineering (2025.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have significantly improved productivity in a number of routine tasks. |
| Approach: | They propose two metrics for classification tasks, namely *sensitivity* and *consistency*, which are complementary to task performance. |
| Outcome: | The proposed metrics are complementary to task performance and can be used to guide prompt engineering and obtain LLMs that balance robustness and performance. |
Counterfactual-Consistency Prompting for Relative Temporal Understanding in Large Language Models (2025.acl-short)
Copied to clipboard
| Challenge: | Existing work has highlighted that large language models lack temporal reasoning abilities, especially when attempting to infer temporal relationships without relying on absolute time indicators. |
| Approach: | They propose a method that generates counterfactual questions and enforces collective constraints, enhancing the model’s consistency. |
| Outcome: | The proposed method shows significant improvements in event ordering for explicit and implicit events and temporal commonsense understanding. |
Can Language Models Follow Multiple Turns of Entangled Instructions? (2025.findings-emnlp)
Copied to clipboard
Chi Han, Xin Liu, Haodong Wang, Shiyang Li, Jingfeng Yang, Haoming Jiang, Zhengyang Wang, Qingyu Yin, Liang Qiu, Changlong Yu, Yifan Gao, Zheng Li, Bing Yin, Jingbo Shang, Heng Ji
| Challenge: | Despite of significant achievements in improving instruction-following capabilities of large language models, the ability to process multiple potentially entangled or conflicting instructions remains a considerable challenge. |
| Approach: | They construct multi-turn instruction with 1.1K high-quality multi-turned conversations using the human-in-the-loop approach and examine their capabilities. |
| Outcome: | The proposed model shows that it is difficult to integrate multiple turns and balance competing objectives when instructions intersect or conflict. |