Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering (D19-1)
Copied to clipboard
| Challenge: | Existing studies have shown that BERT models can find answers from multiple passages . however, the results of these studies are still unaddressed. |
| Approach: | They propose a multi-passage BERT model to globally normalize answer scores across all passages of the same question. |
| Outcome: | The proposed model outperforms state-of-the-art models on four benchmarks. |
Similar Papers
Towards Interpreting BERT for Reading Comprehension Based QA (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained language models such as ELMO and XLNet have achieved state-of-the-art performance on various NLP tasks. |
| Approach: | They propose to define a layer’s role or functionality using Integrated Gradients and perform preliminary analysis across all layers. |
| Outcome: | The proposed model performs better than existing models on RCQA and ELMO, but it lacks the human-level performance needed to perform the task. |
A Primer in BERTology: What We Know About How BERT Works (2020.tacl-1)
Copied to clipboard
| Challenge: | a new study examines the current state of knowledge about the BERT model . the model is a stack of transformer encoder layers that are based on multiple self-attention ''heads'' |
| Approach: | They present a survey of over 150 studies of the popular Transformer-based model BERT . they discuss the current state of knowledge about how BERT works and how it is represented . |
| Outcome: | The proposed model is based on the Transformer-based model with state-of-the-art results . the proposed model has little cognitive motivation and is too small to perform ablation studies . |
Simple yet Effective Bridge Reasoning for Open-Domain Multi-Hop Question Answering (D19-58)
Copied to clipboard
| Challenge: | Existing work on open-domain multi-hop question answering relies on off-the-shelf information retrieval techniques to retrieve answer passages. |
| Approach: | They propose a new subproblem for open-domain multi-hop question answering . they aim to recognize the anchor from a set of start passages with a reading comprehension model . |
| Outcome: | The proposed method significantly improves the baseline method on the open-domain hotpotQA benchmark. |
Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing approaches to extracting answer from text are expensive to train and train. |
| Approach: | They investigate how much models benefit from retrieving text passages . they obtain state-of-the-art results on the Natural Questions and TriviaQA open benchmarks ." |
| Outcome: | The proposed model performs better when retrieving more passages than previously thought . |
Question Answering Using Hierarchical Attention on Top of BERT Features (D19-58)
Copied to clipboard
| Challenge: | Recent advances in QA models focus on the targeted area in the passage. |
| Approach: | They propose a model which uses BERT and hierarchical attention to locate a continuous span of the passage that is the answer to the question. |
| Outcome: | The proposed model is based on a BERT embedding and a hierarchical attention model . it can locate a continuous span of the passage that is the answer to the question . |
BERT-QE: Contextualized Query Expansion for Document Re-ranking (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to expand query use pseudo relevance feedback (PRF) but they are under-equipped to evaluate the relevance of information pieces used for expansion. |
| Approach: | They propose a query expansion model that leverages the BERT model to select relevant document chunks for expansion. |
| Outcome: | The proposed model significantly outperforms existing models on the TREC Robust04 and GOV2 test collections. |
BERT-kNN: Adding a kNN Search Component to Pretrained Language Models for Better QA (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained language models (PLMs) capture a diverse range of linguistic and factual knowledge without the use of finetuning. |
| Approach: | They combine BERT with a traditional information retrieval step and a kNN search to improve the recall of facts encountered during training. |
| Outcome: | The proposed approach outperforms baselines on cloze-style QA by large margins without further training. |
Open-Domain Question Answering (2020.acl-tutorials)
Copied to clipboard
| Challenge: | tutorial provides a comprehensive overview of cutting-edge research in open-domain question answering (QA) |
| Approach: | tutorial provides a comprehensive overview of cutting-edge research in open-domain question answering . focus will shift to cutting- edge models proposed for open- domain QA . |
| Outcome: | The tutorial will cover cutting-edge research in open-domain question answering (QA) it will cover two-stage retriever-reader approaches, dense retriever and end-to-end training, and retriever free methods . |
Cross-Domain Modeling of Sentence-Level Evidence for Document Retrieval (D19-1)
Copied to clipboard
| Challenge: | Existing test collections provide only document-level relevance judgments, and documents exceed the length that BERT was designed to handle. |
| Approach: | They propose to aggregate sentence-level evidence to rank news articles using BERT . they also leverage passage-level relevance judgments available in other domains to fine-tune BERT models that capture cross-domain notions of relevance. |
| Outcome: | The proposed model aggregates sentence-level evidence to rank documents on three standard test collections. |
What’s so special about BERT’s layers? A closer look at the NLP pipeline in monolingual and multilingual models (2020.findings-emnlp)
Copied to clipboard
| Challenge: | In addition, information on part-of-speech tagging is spread over different parts of the network and the pipeline might not be as neat as it seems. |
| Approach: | They propose to probe Dutch BERT-based model and multilingual BERT model for Dutch NLP tasks to see if this holds true for other languages. |
| Outcome: | The proposed model is based on a Dutch model and a multilingual model for Dutch NLP tasks. |