| Challenge: | Recent work in open-domain question answering (QA) has explored generating context passages from large language models (LLMs) however, it is not well understood why generated passages can be more effective than retrieved ones. |
| Approach: | They propose to generate context passages from large language models by paraphrasing human-annotated gold context using LLMs to observe knowledge corpus error. |
| Outcome: | The proposed framework shows that paraphrasing human-annotated gold contexts improves performance over retrieval steps. |
Similar Papers
DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text (2024.findings-naacl)
Copied to clipboard
| Challenge: | Retrievalaugmented LLMs have been used to ground LLM in external knowledge . a gap exists in the current landscape regarding the effectiveness of grounding LLM on heterogeneous knowledge sources. |
| Approach: | They propose a model that uses symbolic language to generate symbolic queries . they use a dataset that is generated using predefined reasoning chains and human annotation . |
| Outcome: | The proposed model outperforms previous approaches by a significant margin in QA tasks over text. |
Knowledge Extraction on Semi-Structured Content: Does It Remain Relevant for Question Answering in the Era of LLMs? (2026.eacl-long)
Copied to clipboard
Kai Sun, Yin Huang, Srishti Mehra, Mohammad Kachuee, Xilun Chen, Renjie Tao, Zhaojiang Lin, Andrea Jessee, Nirav Shah, Alex L Betty, Yue Liu, Anuj Kumar, Wen-tau Yih, Xin Luna Dong
| Challenge: | Existing literature on knowledge extraction for question answering questions whether it is still relevant for question answerrs. |
| Approach: | They extend an existing benchmark with knowledge extraction annotations and evaluate commercial and open-source LLMs of varying sizes. |
| Outcome: | The proposed model can achieve high QA accuracy, but can still benefit from knowledge extraction through augmentation with extracted triples and multi-task learning. |
How Accurate Are LLMs at Multi-Question Answering on Conversational Transcripts? (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are used for question answering over long contexts . high computational costs and latency hinder the process . |
| Approach: | They explore the capabilities of Large Language Models to answer multiple questions based on the same conversational context. |
| Outcome: | The proposed models outperform proprietary and public models in question answering . their results show that they can be cost-effective and transparent . |
Natural Context Drift Undermines the Natural Language Understanding of Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | generative Large Language Models (LLMs) are based on natural text evolution . |
| Approach: | They propose a framework for curating naturally evolved variants of reading passages from contemporary QA benchmarks and for analysing LLM performance across a range of semantic similarity scores. |
| Outcome: | The proposed framework evaluates QA datasets and LLMs with publicly available training data. |
Evaluating Open-Domain Question Answering in the Era of Large Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Existing evaluation models fail to identify lexical matching failures for open-domain question answering. |
| Approach: | They manually evaluate open-domain QA models by manually evaluating their answers on a popular benchmark. |
| Outcome: | The proposed model performs better on NQ-open than existing models and more than 50% of lexical matching failures are attributed to semantically equivalent answers. |
How Proficient Are Large Language Models in Formal Languages? An In-Depth Insight for Knowledge Base Question Answering (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have validated that large language models (LLMs) are capable of solving some KBQA problems, but there has been little discussion on the differences in LLMs’ proficiency in formal languages used in semantic parsing. |
| Approach: | They propose to evaluate the understanding and generation ability of large language models (LLMs) to deal with differently structured logical forms by examining the inter-conversion of natural and formal language through in-context learning of LLMs. |
| Outcome: | The proposed model can understand formal languages as well as humans, but generating correct logical forms remains a challenge. |
Shortcomings of Question Answering Based Factuality Frameworks for Error Localization (2023.eacl-main)
Copied to clipboard
| Challenge: | Abstractive summarization systems often generate summaries with factual errors . many approaches to detect these errors have been proposed, but this capability has not been evaluated in past research . |
| Approach: | They propose to use question answering-based factuality metrics to detect errors in summaries . they find that QA-based frameworks fail to correctly identify error spans in generated summary . |
| Outcome: | The proposed methods outperform trivial exact match baselines in localizing errors in summaries. |
Can NLI Models Verify QA Systems’ Predictions? (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent question answering systems perform well on benchmark datasets, but are not always well-calibrated to spot spurious answers under distribution shifts. |
| Approach: | They propose to use natural language inference to verify whether answers are correct . they leverage large pre-trained models and recent prior datasets to construct powerful question conversion and decontextualization modules. |
| Outcome: | The proposed approach improves the confidence estimation of a QA model across different domains, evaluated in a selective QA setting. |
Improving Question Answering with External Knowledge (D19-58)
Copied to clipboard
| Challenge: | ARC-Easy, ARC Challenge, and OpenBookQA use Wikipedia to augment training data . performance degrades when additional instances exhibit higher difficulty than original training data. |
| Approach: | They propose two methods for exploiting external knowledge for QA in science . they enrich the original corpus with relevant text snippets from an open-domain resource . the second method simply increases the amount of training data by appending additional in-domain instances. |
| Outcome: | The proposed methods achieve gains in accuracy of 8.1%, 13.0%, and 12.8% on science QA tasks. |
Large Language Models Meet Knowledge Graphs for Question Answering: Synthesis and Opportunities (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown remarkable performance on question-answering tasks due to their superior capabilities in natural language understanding and generation. |
| Approach: | They propose a structured taxonomy that categorizes the methodology of synthesizing LLMs and knowledge graphs for QA according to the categories of QA and the KG’s role when integrating with LLM. |
| Outcome: | The proposed taxonomy categorizes the methods according to the categories of QA and the KG’s role when integrating with LLMs. |