User Feedback in Human-LLM Dialogues: A Lens to Understand Users But Noisy as a Learning Signal (2025.emnlp-main)
Copied to clipboard
| Challenge: | a recent study shows that asking for direct user feedback can be disruptive . we examine whether incorporating the contents of user feedback improves model performance . |
| Approach: | They analyze user feedback in the user-LLM conversation logs and harvest learning signals from it. |
| Outcome: | The proposed approach can lead to model degradation on two user-LM interaction datasets. |
Similar Papers
WildFeedback: Aligning LLMs With In-situ User Interactions And Feedback (2026.acl-long)
Copied to clipboard
Taiwei Shi, Zhuoer Wang, Longqi Yang, Ying-Chun Lin, Zexue He, Mengting Wan, Pei Zhou, Sujay Kumar Jauhar, Sihao Chen, Shan Xia, Hongfei Zhang, Jieyu Zhao, Xiaofeng Xu, Xia Song, Jennifer Neville
| Challenge: | Traditional alignment methods rely on human annotations and are subjective and misalignment with real-world user preferences. |
| Approach: | They propose a framework that leverages in-situ user feedback during conversations with LLMs to create preference datasets automatically. |
| Outcome: | The proposed framework identifies and classifies user feedback to LLM responses between conversation turns and creates examples of preferred and dispreferred responses according to user preferences. |
The Past, Present and Better Future of Feedback Learning in Large Language Models for Subjective Human Preferences and Values (2023.emnlp-main)
Copied to clipboard
| Challenge: | Incorporating human feedback into Large Language Models is a welcome development, but it introduces new biases and challenges. |
| Approach: | They propose to survey 95 articles that use human feedback to steer, guide or tailor the behaviours of large language models. |
| Outcome: | The proposed approaches are based on 95 articles primarily from the ACL and arXiv repositories and highlight five unresolved conceptual and practical challenges. |
Real or Robotic? Assessing Whether LLMs Accurately Simulate Qualities of Human Responses in Human-LLM Dialogue (2026.findings-acl)
Copied to clipboard
Jonathan Ivey, Shivani Kumar, Jiayu Liu, Hua Shen, Sushrita Rakshit, Rohan Raju, Haotian Zhang, Aparna Ananthasubramaniam, Junghwan Kim, Bowen Yi, Dustin Wright, Abraham Israeli, Anders Giovanni Møller, Lechen Zhang, David Jurgens
| Challenge: | Recent work has sought to use large language models to simulate human-human and human-LLM interactions. |
| Approach: | They use a large-scale dataset to generate a paired LLM-LLM and human-LLm dialogues from the WildChat dataset and quantify how well they align with their human counterparts. |
| Outcome: | The proposed models perform similarly in simulating English, Chinese, and Russian dialogues. |
Leveraging Implicit Feedback from Deployment Data in Dialogue (2024.eacl-short)
Copied to clipboard
| Challenge: | Xu et al., 2023) and Bai ed., 2019) use crowdworkers to collect signals from natural dialogue episodes. |
| Approach: | They use the publicly released BlenderBot deployment data to extract signals from conversations to implicitly measure the quality of a machine-generated utterance. |
| Outcome: | The proposed model improves over baseline models, but some proxy signals can lead to undesirable generations. |
Towards Implicit Bias Detection and Mitigation in Multi-Agent LLM Interactions (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that large language models are susceptible to societal biases due to their exposure to human-generated data. |
| Approach: | They propose two strategies to mitigate implicit gender biases in large language models . they create scenarios where implicit gender is present and develop a metric to assess the presence of biase . |
| Outcome: | The proposed methods mitigate implicit biases with self-reflection and fine-tuning. |
Can Neural Machine Translation be Improved with User Feedback? (N18-3)
Copied to clipboard
| Challenge: | a recent study has focused on the use of explicit and implicit feedback for neural machine translation (NMT) a new study uses explicit and implied feedback to improve performance of NMT with human reinforcement. |
| Approach: | They propose to use real logged feedback to improve neural machine translation with human reinforcement. |
| Outcome: | The proposed method improves translation quality metrics with implicit task-based feedback . the proposed method is based on explicit and implicit feedback collected on the eBay platform . |
Reading Between the Prompts: How Stereotypes Shape LLM’s Implicit Personalization (2025.emnlp-main)
Copied to clipboard
| Challenge: | Prior work has shown that such inferences can lead to lower quality responses for users assumed to be from minority groups. |
| Approach: | They analyze LLMs' latent user representations through both model internals and generated answers to targeted user questions. |
| Outcome: | The proposed models infer demographic attributes based on stereotypical signals, which persists even when the user explicitly identifies with a different demographic group. |
I Learn Better If You Speak My Language: Understanding the Superior Performance of Fine-Tuning Large Language Models with LLM-Generated Responses (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent research has demonstrated that a large language model (LLM) can generate training data for another LLM, or for creating supplementary training materials, such as rationales. |
| Approach: | They conduct an in-depth investigation to understand why fine-tuning an LLM with responses generated by a LLM often yields better results than using responses generated from humans. |
| Outcome: | The proposed approach can be used to transfer knowledge from a larger model to a smaller one, or for creating supplementary training materials, such as rationales. |
LETI: Learning to Generate from Textual Interactions (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing techniques fine-tune on input-output pairs or with numerical rewards that gauge the output quality are not effective. |
| Approach: | They propose to fine-tune pre-trained language models with binary labels and a Python interpreter to get textual feedback from the inputs. |
| Outcome: | The proposed model outperforms the base model on unseen problems and achieves comparable or better performance on humanEval. |
Toward Beginner-Friendly LLMs for Language Learning: Controlling Difficulty in Conversation (2026.findings-eacl)
Copied to clipboard
| Challenge: | Practicing conversations with large language models is a promising alternative to traditional in-person language learning. |
| Approach: | They propose a new token-level evaluation metric, Token Miss Rate, that measures the proportion of incomprehensible tokens per utterance and correlates strongly with human judgments. |
| Outcome: | The proposed methods improve comprehensibility for beginner speakers from 39.4% to 83.3%, compared with prompting alone and a token-level evaluation metric, Token Miss Rate (TMR). |