| Challenge: | Current textual question answering models fail to generalize to out-of-domain settings. |
| Approach: | They propose to decompose question and context into smaller units and align them to find the answer. |
| Outcome: | The proposed model is more robust than the standard BERT QA model on adversarial and out-of-domain datasets. |
Similar Papers
What do Models Learn from Question Answering Datasets? (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models have outperformed humans on question answering datasets, but they have yet to outperform humans on the task of question answering itself. |
| Approach: | They evaluate BERT-based question answering models on their generalizability to out-of-domain examples, responses to missing or incorrect data, and ability to handle question variations. |
| Outcome: | The proposed models outperform human baselines on the widely-used SQuAD 1.1 and SQu AD 2.0 datasets. |
Retrieving Support to Rank Answers in Open-Domain Question Answering (2025.emnlp-main)
Copied to clipboard
| Challenge: | a novel question answering architecture retrieves content relevant to the combined pair . previous work on automatic claim verification has shown hallucinations . |
| Approach: | They propose a question-answer architecture that prioritizes supporting evidence . it retrieves paragraphs that directly substantiate the correctness of a with respect to q . |
| Outcome: | The proposed approach can be used by large language models to retrieve explanatory paragraphs that ground their reasoning. |
Bend but Don’t Break? Multi-Challenge Stress Test for QA Models (D19-58)
Copied to clipboard
| Challenge: | a gap remains in reasoning ability compared to a human, and performance tends to degrade when models are exposed to less-constrained tasks. |
| Approach: | They conduct extensive qualitative and quantitative analyses on the results of four models across four datasets . they relate common errors to model capabilities and discuss a way forward . |
| Outcome: | The proposed model performance is based on the results of four models across four datasets. |
Alignment over Heterogeneous Embeddings for Question Answering (N19-1)
Copied to clipboard
| Challenge: | Existing approaches for non-factoid question answering are based on heterogeneous embeddings that model text at different levels of abstraction. |
| Approach: | They propose a fast, mostly-unsupervised approach for non-factoid question answering called Alignment over Heterogeneous Embeddings (AHE) it aligns each word in the question and candidate answer with the most similar word in retrieved supporting paragraph and a meta-classifier that learns how much to trust the predictions over each representation. |
| Outcome: | The proposed approach outperforms other supervised approaches on the AI2 Reasoning Challenge dataset and the WikiQA dataset. |
Open-Domain Question Answering (2020.acl-tutorials)
Copied to clipboard
| Challenge: | tutorial provides a comprehensive overview of cutting-edge research in open-domain question answering (QA) |
| Approach: | tutorial provides a comprehensive overview of cutting-edge research in open-domain question answering . focus will shift to cutting- edge models proposed for open- domain QA . |
| Outcome: | The tutorial will cover cutting-edge research in open-domain question answering (QA) it will cover two-stage retriever-reader approaches, dense retriever and end-to-end training, and retriever free methods . |
Joint Models for Answer Verification in Question Answering Systems (2021.acl-long)
Copied to clipboard
| Challenge: | Using a joint approach, we found that the model is more efficient than those developed in machine reading (MR) work. |
| Approach: | They propose a joint model for selecting correct answer sentences among the top k provided by answer sentence selection modules. |
| Outcome: | The proposed model improves on WikiQA, TREC-QA, and a real-world dataset. |
Exploring The Landscape of Distributional Robustness for Question Answering Models (2022.findings-emnlp)
Copied to clipboard
Anas Awadalla, Mitchell Wortsman, Gabriel Ilharco, Sewon Min, Ian Magnusson, Hannaneh Hajishirzi, Ludwig Schmidt
| Challenge: | Existing methods for predicting distributional robustness fail to generalize reliably in a variety of test conditions. |
| Approach: | They conduct a large empirical evaluation to investigate the landscape of distributional robustness in question answering. |
| Outcome: | The proposed methods are more robust to distribution shifts than fully fine-tuned models, and few-shot prompt models exhibit better robustness than few- shot prompt models. |
QuASE: Question-Answer Driven Sentence Encoding (2020.acl-main)
Copied to clipboard
| Challenge: | Question-answering (QA) data often encodes essential information in many facets . a growing interest of QA has led to many large-scale QA datasets available to the community . |
| Approach: | They propose a question-answer driven sentence encoding framework to learn representations from QA data. |
| Outcome: | The proposed framework learns representations from QA data, using BERT or other state-of-the-art contextual language models. |
Can NLI Models Verify QA Systems’ Predictions? (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent question answering systems perform well on benchmark datasets, but are not always well-calibrated to spot spurious answers under distribution shifts. |
| Approach: | They propose to use natural language inference to verify whether answers are correct . they leverage large pre-trained models and recent prior datasets to construct powerful question conversion and decontextualization modules. |
| Outcome: | The proposed approach improves the confidence estimation of a QA model across different domains, evaluated in a selective QA setting. |
QAEncoder: Towards Aligned Representation Learning in Question Answering Systems (2025.acl-long)
Copied to clipboard
Zhengren Wang, Qinhan Yu, Shida Wei, Zhiyu Li, Feiyu Xiong, Xiaoxing Wang, Simin Niu, Hao Liang, Wentao Zhang
| Challenge: | Modern QA systems entail retrieval-augmented generation (RAG) for accurate and trustworthy responses, but the inherent gap between user queries and relevant documents hinders precise matching. |
| Approach: | They propose a retrieval-augmented generation (RAG)-based approach to bridge this gap by attaching document fingerprints to the embedding to estimate the expectation of potential queries. |
| Outcome: | Experiments across diverse datasets, languages, and embedding models confirm the proposed solution is simple-yet-effective with zero additional index storage, retrieval latency, training costs, or catastrophic forgetting and hallucination issues. |