PreWoMe: Exploiting Presuppositions as Working Memory for Long Form Question Answering (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to LFQA are limited to information-seeking questions . however, users' questions in the real-world may often mislead language models (LMs) to output misinformation. |
| Approach: | They propose a unified approach capable of handling any type of information-seeking question. |
| Outcome: | The proposed approach can handle any type of information-seeking question. |
Similar Papers
A Critical Evaluation of Evaluations for Long-form Question Answering (2023.acl-long)
Copied to clipboard
| Challenge: | Long-form question answering (LFQA) is an emerging research area within QA . however, its flexibility poses enormous challenges for evaluation . |
| Approach: | They conduct the first targeted study of the evaluation of long-form answers, covering both human and automatic evaluation practices. |
| Outcome: | The proposed evaluations cover human and automatic evaluations. |
CREPE: Open-Domain Question Answering with False Presuppositions (2023.acl-long)
Copied to clipboard
| Challenge: | Existing question answering datasets assume all questions have well defined answers. |
| Approach: | They propose a QA dataset containing a distribution of false presuppositions . they find that 25% of questions contain false presumptions . |
| Outcome: | The proposed model finds that 25% of questions contain false presuppositions . the model can find presuffpositions moderately well, but struggle when predicting correctness . |
Hurdles to Progress in Long-form Question Answering (2021.naacl-main)
Copied to clipboard
| Challenge: | Long-form question answering (LFQA) involves retrieving documents relevant to a given question and using them to generate a paragraph-length answer. |
| Approach: | They propose a long-form question answering system that relies on sparse attention and contrastive retriever learning to achieve state-of-the-art performance on the ELI5 LFQA dataset. |
| Outcome: | The proposed system tops the public leaderboard on the ELI5 LFQA dataset, but it has several troubling issues. |
Which Linguist Invented the Lightbulb? Presupposition Verification for Question-Answering (2021.acl-long)
Copied to clipboard
| Challenge: | Existing Question-Answering (QA) datasets contain unanswerable questions . however, their treatment in QA systems remains primitive . |
| Approach: | They propose a framework that provides answers based on presupposition failure over oracle behavior of existing QA systems. |
| Outcome: | The proposed system provides responses based on presupposition failure over oracle behavior of existing QA systems. |
Evaluating Reasoning Models for Queries with Presuppositions (2026.findings-acl)
Copied to clipboard
| Challenge: | Prior work notes that large language models fail to challenge erroneous assumptions and can reinforce users’ misinformed opinions. |
| Approach: | They construct queries with varying degrees of presuppositions spanning health, science, and general knowledge and evaluate several widely-deployed models. |
| Outcome: | The proposed models achieve higher accuracy but fail to challenge a large fraction of false presuppositions. |
Read before Generate! Faithful Long Form Question Answering with Machine Reading (2022.findings-acl)
Copied to clipboard
| Challenge: | Long-form question answering (LFQA) generates a paragraph-length answer for a given question. |
| Approach: | They propose a framework that jointly models answer generation and machine reading. |
| Outcome: | The proposed model generates a more factually accurate answer from millions of documents retrieved from a large dataset. |
Augmenting Pre-trained Language Models with QA-Memory for Open-Domain Question Answering (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for open-domain question-answering use an open book approach . a recent alternative is to retrieve from a collection of previously-generated question-annwer pairs . |
| Approach: | They propose a new QA system that augments a text-to-text model with a large memory of question-answer pairs and a task for the latent step of question retrieval. |
| Outcome: | The proposed system outperforms closed-book QA and can answer multi-hop questions. |
Retrieving Contextual Information for Long-Form Question Answering using Weak Supervision (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing retrievers for long-form question answering are optimized for information that directly targets the question, missing out on contextual information. |
| Approach: | They propose to use weak supervision techniques to optimize retrieval for contextual information. |
| Outcome: | The proposed techniques improve the end-to-end QA performance on a conversational QA dataset. |
KnowMe-Bench: Benchmarking Person Understanding for Lifelong Digital Companions (2026.acl-long)
Copied to clipboard
Tingyu Wu, Zhisheng Chen, Ziyan Weng, Shuhe Wang, Shuo Zhang, Sen Hu, Silin Wu, Qizhen Lan, Huacan Wang, Ronghao Chen
| Challenge: | Existing long-horizon memory benchmarks use multi-turn dialogues or synthetic user histories . despite rapid progress on long-term memory evaluation, there are gaps in existing benchmarks . |
| Approach: | They propose a long-form autobiographical narrative benchmark that reconstructs each narrative into a flashback-aware, time-anchored stream and evaluates models with evidence-linked questions. |
| Outcome: | The proposed benchmarks build from long-form autobiographical narratives . they show that retrieval-augmented systems improve factual accuracy while errors persist on temporally grounded explanations and higher-level inferences. |
WebCPM: Interactive Web Search for Chinese Long-form Question Answering (2023.acl-long)
Copied to clipboard
Yujia Qin, Zihan Cai, Dian Jin, Lan Yan, Shihao Liang, Kunlun Zhu, Yankai Lin, Xu Han, Ning Ding, Huadong Wang, Ruobing Xie, Fanchao Qi, Zhiyuan Liu, Maosong Sun, Jie Zhou
| Challenge: | Long-form question answering requires two procedures: information retrieval and information synthesis. |
| Approach: | They propose a Chinese long-form question answering dataset called WebCPM . the dataset is based on a web search interface that engages with a search engine in real time . |
| Outcome: | The proposed dataset generates answers that are no worse than human-written ones . the dataset is the first Chinese LFQA dataset . |