Detecting Ambiguous Utterances in an Intelligent Assistant (2024.emnlp-industry)
Copied to clipboard
| Challenge: | ambiguous utterances can be interpreted as either chat or task intents in intelligent assistants . ambiguity of intent is particularly noticeable in intelligent devices where task-oriented and non-task-oriented utterrances are mixed and most utterations are short due to characteristics of devices. |
| Approach: | They propose to feed sentence embeddings developed from microblogs and search logs with a self-attention mechanism to detect ambiguous utterances robustly. |
| Outcome: | The proposed model outperforms baselines and a strong LLM-based model. |
Similar Papers
Aligning Language Models to Explicitly Handle Ambiguity (2024.emnlp-main)
Copied to clipboard
Hyuhng Joon Kim, Youna Kim, Cheonbok Park, Junyeob Kim, Choonghyun Park, Kang Min Yoo, Sang-goo Lee, Taeuk Kim
| Challenge: | Large language models (LLMs) are not specifically trained to deal with ambiguous utterances . ambiguity can lead to varying interpretations of the same input based on different assumptions or background knowledge . |
| Approach: | They propose a pipeline that aligns large language models to manage ambiguous queries . they propose to use their own assessment of perceived ambiguity to detect and manage queries a . |
| Outcome: | Experimental results show that APA empowers LLMs to detect and manage ambiguous queries while retaining the ability to answer clear questions. |
Disambiguation in Conversational Question Answering in the Era of LLMs and Agents: A Survey (2025.emnlp-main)
Copied to clipboard
Mehrab Tanjim, Yeonjun In, Xiang Chen, Victor Bursztyn, Ryan A. Rossi, Sungchul Kim, Guang-Jie Ren, Vaishnavi Muppala, Shun Jiang, Yongsung Kim, Chanyoung Park
| Challenge: | Existing literature on ambiguity and disambiguation with Large Language Models (LLMs) ambiguities are a fundamental challenge in human-AI interactions due to complexity and flexibility of human language. |
| Approach: | They propose to define key terms and concepts and categorize various disambiguation approaches enabled by LLMs and provide a comparative analysis of their advantages and disadvantages. |
| Outcome: | The proposed frameworks are compared against different disambiguation approaches and highlight their relevance for future research. |
Semantic Ambiguity Detection in Sentence Classification using Task-Specific Embeddings (2023.acl-industry)
Copied to clipboard
| Challenge: | ambiguity is a major obstacle to providing services based on sentence classification . authors use similarity in a semantic space to detect ambiguities in training data and scenarios . |
| Approach: | They use similarity in a semantic space to detect ambiguities in service scenarios and training data. |
| Outcome: | The proposed approach can detect ambiguities and debug services. |
AmbiK: Dataset of Ambiguous Tasks in Kitchen Environment (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are used for behavior planning given natural language instructions from the user. |
| Approach: | They propose to use a textual dataset of ambiguous instructions addressed to a robot in a kitchen environment to compare them. |
| Outcome: | The proposed dataset includes 1000 pairs of ambiguous tasks and their unambiguous counterparts, with environment descriptions, clarifying questions and answers, user intents, and task plans. |
Clarify When Necessary: Resolving Ambiguity Through Interaction with LMs (2025.findings-naacl)
Copied to clipboard
| Challenge: | Ambiguity is embedded throughout natural language, and even simple utterances can have multiple interpretations when read in isolation. |
| Approach: | They propose a task-agnostic framework for evaluating a system’s ability to determine when to ask for clarification. |
| Outcome: | The proposed framework outperforms existing uncertainty estimation approaches at identifying predictions that will benefit from clarification. |
Model Analysis & Evaluation for Ambiguous Question Answering (2023.findings-acl)
Copied to clipboard
| Challenge: | Ambiguous questions are a challenge for Question Answering models as they require answers that cover multiple interpretations of the original query. |
| Approach: | They aim to investigate whether model/data scaling improves the answers’ quality and whether automated metrics align with human judgment. |
| Outcome: | The proposed models can generate long-form answers that combine conflicting information and provide valuable insights into the limitations of the current approaches. |
We’re Afraid Language Models Aren’t Modeling Ambiguity (2023.emnlp-main)
Copied to clipboard
Alisa Liu, Zhaofeng Wu, Julian Michael, Alane Suhr, Peter West, Alexander Koller, Swabha Swayamdipta, Noah Smith, Yejin Choi
| Challenge: | Ambiguity is an intrinsic feature of natural language, allowing us to anticipate misunderstandings and revise our interpretations as listeners. |
| Approach: | They use AmbiEnt to capture ambiguity in a sentence and analyze it to evaluate pretrained LMs. |
| Outcome: | The proposed model can flag political claims in the wild that are misleading due to ambiguity. |
CondAmbigQA: A Benchmark and Dataset for Conditional Ambiguous Question Answering (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) generate unreliable responses due to their cognitive alignment of context and intent. |
| Approach: | They propose a benchmark to identify possible implicit assumptions in QA questions . they use retrieved Wikipedia fragments to identify interpretations for a given query . |
| Outcome: | The proposed benchmark identifies possible implicit assumptions and improves answer accuracy by 11.75% . retrieved Wikipedia fragments help identify possible interpretations for a given query . |
Aligning Predictive Uncertainty with Clarification Questions in Grounded Dialog (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Previous work bases the timing of questions on supervised models learned from interactions between humans. |
| Approach: | They propose to ground the need for questions in the acting agent's predictive uncertainty by using the T5 encoder-decoder architecture to solve a Minecraft Collaborative Building task. |
| Outcome: | The proposed model can detect ambiguous instructions and predict responses better than previous models. |
Intent Discovery for Enterprise Virtual Assistants: Applications of Utterance Embedding and Clustering to Intent Mining (2022.naacl-industry)
Copied to clipboard
| Challenge: | Existing approaches to clustering unlabeled utterances are based on transformerbased sentence embedding methods. |
| Approach: | They propose a semantic embedding approach that can be leveraged to identify clusters of utterances that correspond to unhandled intents. |
| Outcome: | The proposed approach can identify clusters of utterances that correspond to unhandled intents from a large collection of enterprise virtual assistant data using a multi-task softmax loss. |