Measuring Bargaining Abilities of LLMs: A Benchmark and A Buyer-Enhancement Method (2024.findings-acl)
Copied to clipboard
| Challenge: | Using a novel approach, we can evaluate an agent’s bargaining abilities as an asymmetric incomplete information game. |
| Approach: | They propose an approach that integrates a deterministic Offer Generator and an LLM Narrator to create natural language sentences for generated offers. |
| Outcome: | The proposed approach improves the buyer’s deal rates from 26.67% to 88.88% and brings a ten times multiplication of profits on all baselines, even a model that has not been aligned. |
Similar Papers
MERIT Feedback Elicits Better Bargaining in LLM Negotiators (2026.acl-long)
Copied to clipboard
| Challenge: | Empirical results indicate that baseline LLM strategies diverge from human preferences, while our mechanism substantially improves negotiation performance. |
| Approach: | They propose a utility feedback centric framework that measures human-aligned, economically grounded metrics that implicitly measure how well the negotiation aligns with human preference. |
| Outcome: | The proposed framework significantly improves negotiation performance, yielding deeper strategic behavior and stronger opponent awareness. |
Are LLMs Effective Negotiators? Systematic Evaluation of the Multifaceted Capabilities of LLMs in Negotiation Dialogues (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly being utilized as AI negotiation agents . however, prior research on LLMs lacks a systematic evaluation of their diverse capabilities in negotiation. |
| Approach: | They propose to analyze the multifaceted capabilities of Large Language Models (LLMs) across diverse dialogue scenarios throughout the stages of a typical negotiation interaction. |
| Outcome: | The proposed model outperforms GPT-4 in many negotiation tasks while identifying specific challenges, such as making subjective assessments and generating contextually appropriate, strategically advantageous responses. |
Enhancing Open-Domain Task-Solving Capability of LLMs via Autonomous Tool Integration from GitHub (2025.acl-long)
Copied to clipboard
Bohan Lyu, Xin Cong, Heyang Yu, Pan Yang, Cheng Qian, Zihe Wang, Yujia Qin, Yining Ye, Yaxi Lu, Chen Qian, Zhong Zhang, Yukun Yan, Yankai Lin, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing approaches lack flexibility to address diverse and ever-evolving user queries in open domains. |
| Approach: | They propose to evaluate LLMs on open-domain knowledge that requires tools to solve diverse and ever-evolving user queries. |
| Outcome: | The proposed system outperforms baselines in the open domain task-solving benchmark. |
ACE: A LLM-based Negotiation Coaching System (2024.emnlp-main)
Copied to clipboard
| Challenge: | The rapid progress of LLMs has led to the development of more sophisticated AI tutoring systems. |
| Approach: | They develop an LLM-based assistant for coaching negotiation that provides users with targeted feedback for improvement. |
| Outcome: | The proposed system improves negotiation performance significantly compared to a system that doesn’t provide feedback and one which uses an alternative method. |
Noise, Adaptation, and Strategy: Assessing LLM Fidelity in Decision-Making (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly used for social science simulations . however, most evaluations focus on task optimality rather than variability and adaptation characteristic of human decision-making. |
| Approach: | They propose a process-oriented evaluation framework with progressive interventions to evaluate two economics tasks using large language models. |
| Outcome: | The proposed evaluation framework targets two economic tasks with progressive interventions. |
Aligning Black-box Language Models with Human Judgments (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly used as automated judges to evaluate recommendation systems, search engines, and other subjective tasks. |
| Approach: | They propose a framework to align LLM judgments with individual human evaluators or their aggregated judgments without retraining or fine-tuning the LLM. |
| Outcome: | The proposed framework achieves 142% improvement in agreement across 29 tasks and exceeds inter-human agreement on four out of six tasks. |
A Comprehensive Survey of Process Reward Models: Data Generation, Model Construction, and Usage (2026.acl-long)
Copied to clipboard
Congmin Zheng, Jiachen Zhu, Zhuoying Ou, Yuxiang Chen, Kangning Zhang, Rong Shan, Zeyu Zheng, Mengyue Yang, Jianghao Lin, Yong Yu, Weinan Zhang
| Challenge: | Large Language Models (LLMs) have advanced reasoning ability, yet conventional alignment remains dominated by outcome reward models that judge only final answers. |
| Approach: | They summarize applications across math, code, text, multimodal reasoning, robotics, and agents . goal is to clarify design spaces, reveal open challenges, and guide future research toward fine-grained, robust reasoning alignment. |
| Outcome: | The proposed model enables finer credit assignment, richer diagnostics, and improved robustness. |
PersonaGym: Evaluating Persona Agents and LLMs (2025.findings-emnlp)
Copied to clipboard
Vinay Samuel, Henry Peng Zou, Yue Zhou, Shreyas Chaudhari, Ashwin Kalyan, Tanmay Rajpurohit, Ameet Deshpande, Karthik R Narasimhan, Vishvak Murahari
| Challenge: | Persona agents are LLM agents conditioned to act according to an assigned persona . evaluating how faithfully these agents adhere to their personas remains a challenge . |
| Approach: | a new study evaluates persona agents' ability to act according to an assigned persona . a persona agent's person score is a human-aligned automatic metric that can be used to evaluate a model . |
| Outcome: | a new evaluation framework and a human-aligned automatic metric show that persona agents can perform better. |
Assistive Large Language Model Agents for Socially-Aware Negotiation Dialogues (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have shown that virtual agents can help humans achieve task and social goals. |
| Approach: | They propose a tuning-free and label-free method to identify high-quality ICL exemplars for the remediator agent and propose measurable criteria to measure the quality of the negotiation outcomes. |
| Outcome: | The proposed model is able to improve negotiation outcomes across three negotiation topics. |
AbsInstruct: Eliciting Abstraction Ability from LLMs through Explanation Tuning with Plausibility Estimation (2024.acl-long)
Copied to clipboard
Zhaowei Wang, Wei Fan, Qing Zong, Hongming Zhang, Sehyun Choi, Tianqing Fang, Xin Liu, Yangqiu Song, Ginny Wong, Simon See
| Challenge: | Existing work shows that LLMs are deficient in abstract ability, and how to improve it remains unexplored. |
| Approach: | They propose a framework AbsInstruct to enhance LLMs’ abstract ability through instruction tuning. |
| Outcome: | The proposed framework can enhance LLMs’ abstraction ability with strong generalization performance while maintaining their general instruction-following abilities. |