Madly Ambiguous: A Game for Learning about Structural Ambiguity and Why It’s Hard for Computers (N18-5)
Copied to clipboard
| Challenge: | the game is an open source, in-browser game aimed at teaching audiences of all ages about structural ambiguity and the difficulties it poses for natural language processing. |
| Approach: | the game is an open source, in-browser game aimed at teaching audiences of all ages about structural ambiguity and the difficulties it poses for natural language processing. |
| Outcome: | the game is an open source, in-browser game aimed at teaching audiences of all ages about structural ambiguity and the difficulties it poses for natural language processing. |
Similar Papers
Model Analysis & Evaluation for Ambiguous Question Answering (2023.findings-acl)
Copied to clipboard
| Challenge: | Ambiguous questions are a challenge for Question Answering models as they require answers that cover multiple interpretations of the original query. |
| Approach: | They aim to investigate whether model/data scaling improves the answers’ quality and whether automated metrics align with human judgment. |
| Outcome: | The proposed models can generate long-form answers that combine conflicting information and provide valuable insights into the limitations of the current approaches. |
Tree of Clarifications: Answering Ambiguous Questions with Retrieval-Augmented Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | In open-domain question answering, users often ask ambiguous questions (AQs) . one approach is to identify all possible interpretations of the AQ and generate a long-form answer addressing them all. |
| Approach: | They propose a framework that generates a long-form answer addressing all possible interpretations of an ambiguous question. |
| Outcome: | The proposed framework outperforms baselines on ASQA in a few-shot setup across metrics while surpassing fully-supervised baselines trained on the whole training set in terms of Disambig-F1 and Disambigo-ROUGE. |
Asking Clarification Questions to Handle Ambiguity in Open-Domain QA (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Ambiguous questions persist in open-domain question answering because formulating a precise question with a unique answer is often challenging. |
| Approach: | They propose to ask a clarification question where the user’s response will help identify the interpretation that best aligns with the user's intention. |
| Outcome: | The proposed approach achieves F1 of 61.3, 25.1, and 40.5 on the three tasks, demonstrating the need for further improvements while providing competitive baselines for future work. |
AmbigQA: Answering Ambiguous Open-domain Questions (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing open-domain question answering systems assume questions have a single welldefined answer. |
| Approach: | They propose an open-domain question answering task which involves finding every plausible answer and rewriting the question for each one to resolve the ambiguity. |
| Outcome: | The proposed task is based on a dataset covering 14,042 open-domain questions . it shows that strong models benefit from weakly supervised learning . |
We’re Afraid Language Models Aren’t Modeling Ambiguity (2023.emnlp-main)
Copied to clipboard
Alisa Liu, Zhaofeng Wu, Julian Michael, Alane Suhr, Peter West, Alexander Koller, Swabha Swayamdipta, Noah Smith, Yejin Choi
| Challenge: | Ambiguity is an intrinsic feature of natural language, allowing us to anticipate misunderstandings and revise our interpretations as listeners. |
| Approach: | They use AmbiEnt to capture ambiguity in a sentence and analyze it to evaluate pretrained LMs. |
| Outcome: | The proposed model can flag political claims in the wild that are misleading due to ambiguity. |
Mitigating Boundary Ambiguity and Inherent Bias for Text Classification in the Era of Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | a new text classification framework for large language models addresses the problem of boundary ambiguity and inherent biases in LLMs. |
| Approach: | They propose a two-stage classification framework for large language models to mitigate bottlenecks . their approach uses pairwise comparisons to efficiently narrow down options . |
| Outcome: | The proposed framework reduces the number of options and improves on four datasets. |
Do You Get the Hint? Benchmarking LLMs on the Board Game Concept (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models have achieved impressive progress on many benchmarks, yet they still have fundamental weaknesses. |
| Approach: | They introduce Concept, a word-guessing board game, as a benchmark for probing abductive reasoning. |
| Outcome: | The proposed game is easily solved by humans, but is still very challenging for state-of-the-art LLMs (no model exceeds 40% success rate). |
To Know or Not To Know? Analyzing Self-Consistency of Large Language Models under Ambiguity (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have remarkable performance in a variety of tasks due to factual knowledge accumulated during pre-training. |
| Approach: | They propose an evaluation protocol that disentangles knowing from applying knowledge and test state-of-the-art LLMs on 49 ambiguous entities. |
| Outcome: | The proposed evaluation protocol disentangles knowing from applying knowledge and tests state-of-the-art LLMs on 49 ambiguous entities. |
Answering Ambiguous Questions via Iterative Prompting (2023.acl-long)
Copied to clipboard
| Challenge: | Empirical studies show that AmbigPrompt achieves state-of-the-art or competitive results while using less memory and having a lower inference latency than competing approaches. |
| Approach: | They propose an answering model with a prompting model to address imperfections in open-domain question answering . Empirical studies show AmbigPrompt achieves state-of-the-art or competitive results . |
| Outcome: | The proposed framework improves on two commonly-used open benchmarks and achieves state-of-the-art or competitive results while using less memory and having a lower inference latency. |
RAW-C: Relatedness of Ambiguous Words in Context (A New Lexical Resource for English) (2021.acl-long)
Copied to clipboard
| Challenge: | lexical ambiguity is a problem for NLP, but few tasks evaluate its impact on human intuitions. |
| Approach: | They propose to use contextualized word embeddings to evaluate word meaning . they use a dataset of human relatedness judgments and human estimates of sense dominance . |
| Outcome: | The proposed model matches human intuitions with contextualized embeddings on 112 ambiguous words in context with 672 sentence pairs. |