| Challenge: | Existing work on question answering tasks only provide weak supervision for how the answer should be computed . weak supervision is attractive because it is relatively easy to gather, allowing for large datasets . but weak supervision complicates learning because there are many different spurious ways to derive the correct answer. |
| Approach: | They propose a method to convert question answering tasks into discrete latent variable learning problems with a precomputed set of possible solutions that contains one correct option. |
| Outcome: | The proposed approach outperforms previous methods on six QA tasks and achieves state-of-the-art on five of them. |
Similar Papers
Weakly Supervised Pre-Training for Multi-Hop Retriever (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for weakly supervised multi-hop pretraining require costly human annotation. |
| Approach: | They propose a method for weakly supervised multi-hop retriever pretraining without human efforts by generating vector representations of complex questions and subquestion as weak supervision for pre-training. |
| Outcome: | The proposed method is effective and robust on limited data and computational resources. |
Weakly Supervised Formula Learner for Solving Mathematical Problems (2022.coling-1)
Copied to clipboard
| Challenge: | Existing work suggests a two-phase approach to solving mathematical reasoning tasks . however, its reliance on annotated formulas as intermediate labels throughout its training limited its application. |
| Approach: | They propose a framework that allows models to learn optimal formulas autonomously with weak supervision from the final answers to mathematical problems. |
| Outcome: | The proposed framework outperforms baselines trained on incomplete yet imperfect formula annotations and weakly supervised learning methods on two representative mathematical reasoning datasets. |
Latent Retrieval for Weakly Supervised Open Domain Question Answering (P19-1)
Copied to clipboard
| Challenge: | Recent work on open domain question answering (QA) assumes strong supervision of the supporting evidence and/or assumes a blackbox information retrieval (IR) system to retrieve evidence candidates. |
| Approach: | They propose to jointly learn the retriever and reader from question-answer string pairs and without any IR system. |
| Outcome: | The proposed approach outperforms BM25 on open datasets with a learner and reader by 19 points in exact match. |
Generalizing Question Answering System with Pre-trained Language Model Fine-tuning (D19-58)
Copied to clipboard
| Challenge: | Existing methods focus on improving in-domain performance, leaving open the question of how they can generalize to out-of-domain and unseen RC tasks. |
| Approach: | They propose a multi-task learning framework that learns the shared representation across different tasks and builds on a large pre-trained language model and fine-tuned on multiple RC datasets. |
| Outcome: | The proposed framework improves the BERT-Large baseline by 8.39 and 7.22 respectively. |
A Mutual Information Maximization Approach for the Spurious Solution Problem in Weakly Supervised Question Answering (2021.acl-long)
Copied to clipboard
| Challenge: | Weakly supervised question answering usually has only final answers as supervision signals while correct solutions are not provided. |
| Approach: | They propose to explicitly exploit the semantic correlations between question-answer pairs and predicted answers by maximizing mutual information between question and answer pairs. |
| Outcome: | The proposed method significantly outperforms previous learning methods in terms of task performance and is more effective in training models to produce correct solutions. |
Learning to Generalize for Cross-domain QA (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for QA are hampered by increased training costs . current methods suffer significant performance degradation when applied to out-of-domain examples. |
| Approach: | They propose a method that combines prompting methods and linear probing with fine-tuning strategy, which does not entail additional cost. |
| Outcome: | The proposed method outperforms state-of-the-art baselines with an average increase in F1 score of 4.5%-7.9%. |
Improving Question Answering with External Knowledge (D19-58)
Copied to clipboard
| Challenge: | ARC-Easy, ARC Challenge, and OpenBookQA use Wikipedia to augment training data . performance degrades when additional instances exhibit higher difficulty than original training data. |
| Approach: | They propose two methods for exploiting external knowledge for QA in science . they enrich the original corpus with relevant text snippets from an open-domain resource . the second method simply increases the amount of training data by appending additional in-domain instances. |
| Outcome: | The proposed methods achieve gains in accuracy of 8.1%, 13.0%, and 12.8% on science QA tasks. |
Long Context Question Answering via Supervised Contrastive Learning (2022.naacl-main)
Copied to clipboard
| Challenge: | Long-context question answering tasks often require identifying evidence spans (e.g., sentences) prior work showed that jointly training models to perform evidence extraction and question answering is important for achieving high performance. |
| Approach: | They propose a method for equipping long-context QA models with an additional sequence-level objective for better identification of the supporting evidence. |
| Outcome: | The proposed method exhibits consistent improvements on three different strong long-context transformer models, across two challenging question answering benchmarks – HotpotQA and QAsper. |
Weaker Than You Think: A Critical Look at Weakly Supervised Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Weakly supervised learning is a popular approach for training machine learning models in low-resource settings. |
| Approach: | They propose to use weakly supervised learning to train models with noisy labels from weak sources instead of collecting expensive human annotations. |
| Outcome: | The proposed methods outperform weakly supervised methods on various NLP datasets and tasks on the test sets. |
WSL-DS: Weakly Supervised Learning with Distant Supervision for Query Focused Multi-Document Abstractive Summarization (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods to generate abstractive summarizations are lacking labeled training datasets. |
| Approach: | They propose a weakly supervised approach to generate a strong summary from a set of documents based on a query. |
| Outcome: | The proposed approach sets a new state-of-the-art in terms of evaluation metrics on the Document Understanding Conferences dataset. |