Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaboration (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have shown that ChatGPT has limitations such as failing to ask clarifying questions to ambiguous queries or refusing problematic user requests. |
| Approach: | They propose a Proactive Chain-of-Thought prompting scheme which augments LLMs with the goal planning capability over descriptive reasoning chains to trigger proactivity. |
| Outcome: | The proposed scheme augments LLMs with the goal planning capability over descriptive reasoning chains to trigger the proactivity of LLM-based proactive dialogue systems. |
Similar Papers
Cue-CoT: Chain-of-thought Prompting for Responding to In-depth Dialogue Questions with LLMs (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing LLMs generate responses based on the dialogue context, overlooking the underlying linguistic cues about the user status exhibited in the context. |
| Approach: | They propose a linguistic cue-based chain-of-thoughts method which enhances the LLMs inference with an intermediate reasoning step to find cues exhibited in the dialogue. |
| Outcome: | The proposed method outperforms standard prompting methods on in-depth dialogue questions and linguistic cues exhibited in the context. |
Large Language Models Know Your Contextual Search Intent: A Prompting Framework for Conversational Search (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for understanding users’ contextual search intent show unsatisfactory effectiveness and robustness to handle real conversational search scenarios. |
| Approach: | They propose to use large language models to generate multiple query rewrites and hypothetical responses and to aggregate them into an integrated representation that can robustly represent the user’s real contextual search intent. |
| Outcome: | The proposed framework can generate multiple query rewrites and hypothetical responses and can be used to represent the user’s real contextual search intent. |
ProactiveEval: A Unified Evaluation Framework for Proactive Dialogue Agents (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies on proactive dialogue models focus on domain-specific or task-oriented scenarios, which leads to fragmented evaluations and limits the comprehensive exploration of models’ proactive dialogue abilities. |
| Approach: | They propose a framework for evaluating proactive dialogue capabilities of large language models that decomposes proactive dialogue into target planning and dialogue guidance, establishing evaluation metrics across various domains. |
| Outcome: | The proposed framework decomposes proactive dialogue into target planning and dialogue guidance, establishing evaluation metrics across various domains, and enables automatic generation of diverse and challenging evaluation data. |
Rethinking the Evaluation for Conversational Recommendation in the Era of Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation protocols for large language models (LLMs) are inadequate for conversational recommender systems. |
| Approach: | They propose an evaluation approach based on LLMs that harnesses LLM-based user simulators to evaluate ChatGPT's performance. |
| Outcome: | The proposed evaluation approach can simulate various system-user interaction scenarios. |
Frugal Prompting for Dialog Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) are used in natural language processing tasks with an unrealistic speed and effectiveness. |
| Approach: | They propose more compact ways of providing dialog history information while ensuring good performance and reducing model’s inference-API costs. |
| Outcome: | The proposed models have the optimal usable-information density while maintaining good performance and reducing model’s inference-API costs. |
Active Prompting with Chain-of-Thought for Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods to annotate large language models rely on a fixed set of human-annotated exemplars, which are not always the most effective for different tasks. |
| Approach: | They propose a method to adapt large language models to different tasks with task-specific example prompts (annotated with human-designed CoT reasoning) they introduce several metrics to characterize uncertainty so as to select the most uncertain questions for annotation. |
| Outcome: | The proposed method significantly improves performance on eight complex reasoning tasks. |
A Comparative Analysis of Conversational Large Language Models in Knowledge-Based Text Generation (2024.eacl-short)
Copied to clipboard
| Challenge: | Generating natural language text from graph-structured data is essential for conversational information seeking. |
| Approach: | They conduct an empirical analysis of conversational large language models in generating natural language text from semantic triples using a WebNLG dataset. |
| Outcome: | The proposed models improve their ability to generate natural language text from semantic triples using few-shot prompting, post-processing, and efficient fine-tuning techniques. |
Injecting Salesperson’s Dialogue Strategies in Large Language Models with Chain-of-Thought Reasoning (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent research in dialogue systems focuses on task-oriented (TOD) and open-domain (chit-chat) dialogues. |
| Approach: | They propose to use chit-chat to simulate task-oriented dialogues to train sales agents. |
| Outcome: | The proposed model improves coherence and reduces aggression, improving model learning for sales-customer interactions. |
Improving Large Language Models Function Calling and Interpretability via Guided-Structured Templates (2025.emnlp-main)
Copied to clipboard
Hy Dang, Tianyi Liu, Zhuofeng Wu, Jingfeng Yang, Haoming Jiang, Tao Yang, Pei Chen, Zhengyang Wang, Helen Wang, Huasheng Li, Bing Yin, Meng Jiang
| Challenge: | Large language models (LLMs) have strong reasoning and tool-use capabilities, yet fail in real-world tool-interactions due to incorrect parameterization, poor tool selection, or misinterpretation of user intent. |
| Approach: | They propose a curriculum-inspired framework that leverages structured reasoning templates to guide LLMs through more deliberate step-by-step instructions for generating function calls. |
| Outcome: | The proposed framework reduces tool-use errors and improves interpretability and transparency of tool-using agents. |
Teaching Language Models To Gather Information Proactively (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models are often defaulted to passive responses or narrow clarifications when faced with incomplete or under-specified prompts. |
| Approach: | They propose a new task paradigm where LLMs must identify gaps in context and strategically elicit implicit user knowledge through targeted questions. |
| Outcome: | The proposed framework outperforms o3-mini on evaluation metrics and human annotators favor clarification questions and final outlines. |