| Challenge: | Several QA scenarios and datasets have been introduced over the past few years. |
| Approach: | They conduct extensive experiments to investigate the transferability of knowledge from a source QA dataset to a target dataset using two QA models. |
| Outcome: | The proposed model outperforms the previous best model on TOEFL listening comprehension test by 7% on target datasets. |
Similar Papers
What do Models Learn from Question Answering Datasets? (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models have outperformed humans on question answering datasets, but they have yet to outperform humans on the task of question answering itself. |
| Approach: | They evaluate BERT-based question answering models on their generalizability to out-of-domain examples, responses to missing or incorrect data, and ability to handle question variations. |
| Outcome: | The proposed models outperform human baselines on the widely-used SQuAD 1.1 and SQu AD 2.0 datasets. |
Exploring and Predicting Transferability across NLP Tasks (2020.emnlp-main)
Copied to clipboard
Tu Vu, Tong Wang, Tsendsuren Munkhdalai, Alessandro Sordoni, Adam Trischler, Andrew Mattarella-Micke, Subhransu Maji, Mohit Iyyer
| Challenge: | Recent advances in NLP demonstrate the effectiveness of training large-scale language models and transferring them to downstream tasks. |
| Approach: | They conduct an extensive study of the transferability between 33 NLP tasks across three broad classes of problems. |
| Outcome: | The proposed model can improve performance even with low-data source tasks that differ substantially from the target task. |
Multi-Domain Multilingual Question Answering (2021.emnlp-tutorials)
Copied to clipboard
| Challenge: | Question answering (QA) is one of the most challenging tasks in natural language processing. |
| Approach: | a tutorial examines the state-of-the-art approaches to multi-domain and multilingual QA . they introduce standard benchmarks and discuss out-of the-box training with open-domain QA systems . |
| Outcome: | This tutorial aims to bridge the gap between open-domain and multilingual QA. |
Improving Unsupervised Question Answering via Summarization-Informed Question Generation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Question Generation (QG) is the production of meaningful questions given a set of input passages and corresponding answers. |
| Approach: | They propose a method which uses questions generated heuristically from news summaries as a source of training data for a QG system. |
| Outcome: | The proposed method outperforms previous unsupervised models on three in-domain datasets and three out-of-domain ones. |
What Does My QA Model Know? Devising Controlled Probes Using Expert Knowledge (2020.tacl-1)
Copied to clipboard
| Challenge: | Existing models are far from perfect when assessed at the level of clusters of semantically connected probes, such as all hypernym questions about a single concept. |
| Approach: | They propose a method for automatically building probe datasets from expert knowledge sources, allowing systematic control and a comprehensive evaluation. |
| Outcome: | The proposed model is predisposed to recognize certain types of structural linguistic knowledge, but performance degrades even with a slight increase in the number of “hops” in the underlying taxonomic hierarchy. |
Knowledge Transfer from Answer Ranking to Answer Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies show that Question Answering (QA) based on Answer Sentence Selection (AS2) can be improved by generating an improved answer from the top-k ranked answer sentences. |
| Approach: | They propose to train a GenQA model by transferring knowledge from a trained AS2 model . they use top ranked candidate as the generation target and next k top rated candidates as context . |
| Outcome: | The proposed model outperforms existing models on public and industrial datasets. |
Unsupervised Question Answering by Cloze Translation (P19-1)
Copied to clipboard
| Challenge: | Existing QA datasets only available for limited domains and languages. |
| Approach: | They propose to generate context, question and answer triples in an unsupervised manner and synthesize extractive QA training data automatically. |
| Outcome: | The proposed approach outperforms existing QA models on a common EQA benchmark dataset. |
Source-Free Unsupervised Domain Adaptation for Question Answering via Prompt-Assisted Self-learning (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing SFDA methods focus on the adaptation phase, overlooking the impact of source domain training on model generalizability. |
| Approach: | They propose a source-free domain adaptation approach for Question Answering where a model trained on a domain is adapted to unlabeled target domains without additional source data. |
| Outcome: | The proposed model outperforms existing methods in managing domain gaps and demonstrating greater stability across target domains. |
Few-shot Transfer Learning for Knowledge Base Question Answering: Fusing Supervised Models with In-Context Learning (2024.acl-long)
Copied to clipboard
| Challenge: | Existing Knowledge Base Question Answering (KBQA) architectures are expensive and time-consuming to deploy. |
| Approach: | They propose a KBQA architecture that performs KB-retrieval using multiple source-trained retrievers and re-ranks using an LLM. |
| Outcome: | The proposed architecture outperforms adaptations of SoTA KBQA models when training data is limited. |
Zero-Shot Rationalization by Multi-Task Transfer Learning from Question Answering (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to extract rationales from input text are difficult and impractical. |
| Approach: | They propose a method that leverages multi-task learning and transfer learning to generate rationales through question answering in a zero-shot fashion. |
| Outcome: | The proposed method achieves comparable or even better performance without supervised signal for two benchmark rationalization datasets. |