| Challenge: | Neural models have achieved great success on the task of machine reading comprehension, which are typically trained on hard labels. |
| Approach: | They propose a robust training method for machine reading comprehension models to address label sparseness problem by using three strategies to train models on soft labels. |
| Outcome: | The proposed method improves the baseline model performance and achieves state-of-the-art performance on NewsQA and QUOREF. |
Similar Papers
A Self-Training Method for Machine Reading Comprehension with Soft Evidence Extraction (2020.acl-main)
Copied to clipboard
| Challenge: | Existing models for machine reading comprehension lack evidence labels for training models. |
| Approach: | They propose a method which supervises the evidence extractor with auto-generated evidence labels in an iterative process. |
| Outcome: | The proposed method improves on three MRC tasks on seven datasets. |
DuReader_robust: A Chinese Dataset Towards Evaluating Robustness and Generalization of Machine Reading Comprehension in Real-World Applications (2021.acl-short)
Copied to clipboard
| Challenge: | In order to comprehensively verify the robustness and generalization of MRC models, we construct a real-world Chinese dataset - DuReader_robust . |
| Approach: | They introduce a real-world Chinese dataset to evaluate the robustness and generalization of MRC models from three aspects: over-sensitivity, over-stability and generalisation. |
| Outcome: | The proposed model fails to perform well on the challenge test set and may provide suggestions for future model development. |
Benchmarking Robustness of Machine Reading Comprehension Models (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks only evaluate models' robustness under test-time perturbations or adversarial attacks. |
| Approach: | They propose a model-agnostic benchmark to evaluate models' robustness under adversarial attacks. |
| Outcome: | The proposed model-agnostic benchmark evaluates models under four different types of adversarial attacks. |
Denoising Multi-Source Weak Supervision for Neural Text Classification (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent years have witnessed the rapid development of deep neural networks (DNNs) for text classification problems. |
| Approach: | They propose a label denoiser which estimates the source reliability using a conditional soft attention mechanism and reduces label noise by aggregating rule-annotated weak labels. |
| Outcome: | The proposed model outperforms state-of-the-art methods on sentiment, topic, and relation classifications and achieves comparable performance with fully-supervised methods even without labeled data. |
On the Robustness of Reading Comprehension Models to Entity Renaming (2022.naacl-main)
Copied to clipboard
| Challenge: | SpanBERT model is more robust than RoBERTa, despite having similar accuracy on unperturbed test data. |
| Approach: | They propose a pipeline to replace entity names with names from a variety of sources. |
| Outcome: | The proposed model performs worse when entities are renamed, the authors show . SpanBERT, which is pretrained with span-level masking, is more robust than RoBERTa . |
Learn with Noisy Data via Unsupervised Loss Correction for Weakly Supervised Reading Comprehension (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to filter noise for machine reading comprehension (MRC) are difficult to control and introduce noisy data. |
| Approach: | They propose a hierarchical loss correction strategy to avoid fitting noise and enhance clean supervision signals by using an unsupervisedly fitted Gaussian mixture model and a hard bootstrapping loss method. |
| Outcome: | The proposed methods can help improve models significantly on weakly supervised machine reading comprehension datasets. |
Improving Machine Reading Comprehension with General Reading Strategies (N19-1)
Copied to clipboard
| Challenge: | Recent studies have shown that reading strategies improve comprehension levels for readers lacking adequate prior knowledge. |
| Approach: | They propose three general strategies to improve machine reading comprehension (MRC) by fine-tuning a pre-trained model with strategies and a target task. |
| Outcome: | The proposed models improve non-extractive machine reading comprehension (MRC) on the largest general domain multiple-choice dataset RACE. |
Improving the Robustness of Deep Reading Comprehension Models by Leveraging Syntax Prior (D19-58)
Copied to clipboard
| Challenge: | Recent studies indicate that the current machine reading comprehension systems suffer from weak robustness against adversarial samples. |
| Approach: | They propose to take sentence syntax as the leverage in the answer predicting process and exploit the syntactic elements of a question to improve the generalization and robustness of MRC models. |
| Outcome: | The proposed method improves generalization and robustness against adversarial samples, with performance well-maintained. |
Enhancing Automatic Readability Assessment with Pre-training and Soft Labels for Ordinal Regression (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models do not exploit ordinal nature of difficulty grades and make little effort for initialization to facilitate fine-tuning. |
| Approach: | They propose a readability assessment task that assigns a difficulty grade to a text . they use ordinal regression and pairwise relative text difficulty to train the model . |
| Outcome: | The proposed model outperforms competitive neural models and statistical classifiers on most datasets. |
Noisy Label Regularisation for Textual Regression (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods to regularise noisy labels are ineffective in the face of noisy data. |
| Approach: | They propose a method that regularises noisy labels and prevents error propagation from the input layer. |
| Outcome: | The proposed method regularises noisy labels and improves generalisation performance over real-world human-disagreement annotations and randomly-corrupted and data-augmented labels. |