Is Multihop QA in DiRe Condition? Measuring and Reducing Disconnected Reasoning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models exploit dataset artifacts to produce correct answers without connecting information across multiple facts. |
| Approach: | They formalize disconnected reasoning across subsets of supporting facts to reduce disconnected reasoning . they propose an automatic transformation of existing datasets that reduces disconnected reasoning. |
| Outcome: | The proposed model-agnostic probe reduces disconnected reasoning in a reading comprehension setting. |
Similar Papers
Counterfactual Multihop QA: A Cause-Effect Approach for Reducing Disconnected Reasoning (2023.acl-long)
Copied to clipboard
| Challenge: | Existing QA models rely on shortcuts to provide the true answer, referred to as disconnected reasoning problem. |
| Approach: | They propose a causal-effect approach that exploits true multi-hop reasoning instead of shortcuts. |
| Outcome: | The proposed method achieves 5.8% higher points of its Supps score on hotpotQA through true multihop reasoning. |
Understanding and Improving Zero-shot Multi-hop Reasoning in Generative Question Answering (2022.coling-1)
Copied to clipboard
| Challenge: | Generative question answering (QA) models generate answers to complex questions, but their mechanism for doing so is still poorly understood. |
| Approach: | They decompose multi-hop questions into multiple corresponding single-hop question chains and find marked inconsistency in QA models’ answers on these pairs of ostensibly identical question chains. |
| Outcome: | The proposed models lack zero-shot multi-hop reasoning ability when trained on single-hop questions and on logical forms. |
Understanding Dataset Design Choices for Multi-hop Reasoning (N19-1)
Copied to clipboard
| Challenge: | Existing datasets that explicitly focus on multi-hop reasoning are lacking in learning multi-tasking. |
| Approach: | They propose to use sentence-factored models to solve multi-hop question answering tasks . they find spurious correlations in unmasked versions of WikiHop and HotpotQA . |
| Outcome: | The proposed datasets are used to test models on multi-hop question answering tasks. |
♫ MuSiQue: Multihop Questions via Single-hop Question Composition (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing multihop reasoning benchmarks are largely solvable via shortcuts . a bottom–up approach allows us to create a multihop QA dataset that requires proper multihop thinking. |
| Approach: | They propose a bottom–up approach that selects composable pairs of single-hop questions that are connected and adds stringent filters to the construction process. |
| Outcome: | The proposed approach creates a multihop question answering dataset with 25K 2–4 hop questions. |
Avoiding Reasoning Shortcuts: Adversarial Evaluation, Training, and Model Development for Multi-Hop QA (P19-1)
Copied to clipboard
| Challenge: | In multi-hop question answering, models need to connect multiple pieces of evidence scattered in a long context to answer the question. |
| Approach: | They propose to use a control unit that dynamically attends to the question at different reasoning hops to guide the model's multi-hop reasoning. |
| Outcome: | The proposed model outperforms baseline models but is limited on adversarial test. |
Robustifying Multi-hop QA through Pseudo-Evidentiality Training (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to robustify multi-hop question answering models require expensive annotations. |
| Approach: | They propose a method to supervise answers with right reasoning chains without annotations . they compare answers confidence with and without evidence sentences to generate "pseudo-evidentiality" annotations. |
| Outcome: | The proposed model is accurate and robust in multi-hop reasoning. |
Do Multi-Hop Question Answering Systems Know How to Answer the Single-Hop Sub-Questions? (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing models fail to answer a large portion of sub-questions . Existing systems have achieved super-human performance . |
| Approach: | They propose to use a neural decomposition model to generate sub-questions for a multi-hop question and extract the corresponding sub-answers. |
| Outcome: | The proposed model is based on a hotpotQA dataset with a multi-hop question and sub-answers. |
Is Multi-Hop Reasoning Really Explainable? Towards Benchmarking Reasoning Interpretability (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for multi-hop reasoning are not able to evaluate their interpretability . a recent study found that many paths are unreasonable . |
| Approach: | They propose a framework to evaluate the interpretability of multi-hop reasoning models . they annotate all possible rules and establish a benchmark . |
| Outcome: | The proposed framework outperforms existing models in terms of performance and interpretability. |
Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps (2020.coling-main)
Copied to clipboard
| Challenge: | Existing multi-hop question answering datasets do not provide a complete explanation for the reasoning process from the question to the answer. |
| Approach: | They propose a multi-hop question answering dataset that uses structured and unstructured data to test reasoning skills. |
| Outcome: | The proposed dataset ensures multi-hop reasoning while being challenging for multi-models. |
Low-Resource Generation of Multi-hop Reasoning Questions (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods to generate valid and fluent questions from text are limited and insufficient for training. |
| Approach: | They propose to generate multi-hop reasoning questions from the raw text in a low resource circumstance by deducing over multiple relations on several sentences in the text. |
| Outcome: | The proposed model can be applied to the task of machine reading comprehension and achieve significant performance improvements. |