Papers by Chris DuBois
Combining Compressions for Multiplicative Size Scaling on Natural Language Tasks (2022.coling-1)
Copied to clipboard
| Challenge: | Quantization, knowledge distillation, and magnitude pruning are among the most popular methods for neural network compression in NLP. |
| Approach: | They compare accuracy vs. model size tradeoffs using quantization and distillation methods . they find that pruning provides greater benefit than quantization . |
| Outcome: | The proposed methods reduce model size and can accelerate inference, but their relative benefit and combinatorial interactions have not been rigorously studied. |
Entity-Based Knowledge Conflicts in Question Answering (2021.emnlp-main)
Copied to clipboard
| Challenge: | Knowledge-dependent tasks typically use two sources of knowledge: parametric, learned at training time, and contextual, given as a passage at inference time. |
| Approach: | They propose a method to mitigate over-reliance on parametric knowledge, which minimizes hallucination, and improves out-of-distribution generalization by 4% - 7%. |
| Outcome: | The proposed method minimizes hallucination and improves generalization to evolving information by 4% - 7%. |
On the Transferability of Minimal Prediction Preserving Inputs in Question Answering (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work establishes the presence of short, uninterpretable input fragments that yield high confidence and accuracy in neural models. |
| Approach: | They investigate competing hypotheses for the existence of MPPIs in question answering . they discover a perplexing invariance of MPIs to random training seed, model architecture, pretraining, and training domain. |
| Outcome: | The proposed model performance is higher than comparable short queries. |
An Exploration of Data Augmentation and Sampling Techniques for Domain-Agnostic Question Answering (D19-58)
Copied to clipboard
| Challenge: | XLNet model is domain-agnostic for the MRQA 2019 Shared Task . a negative sampling technique is particularly effective for datasets that include unanswerable questions . |
| Approach: | They develop a domain-agnostic question answering model for the MRQA 2019 Shared Task . they use large pre-trained language models, various data sampling strategies and query and context paraphrases generated by back-translation . |
| Outcome: | The proposed model achieves second best Exact Match and F1 in the MRQA leaderboard competition. |
How Effective is Task-Agnostic Data Augmentation for Pretrained Transformers? (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Task-agnostic data augmentations have proven widely effective in computer vision, even on pretrained models. |
| Approach: | They examine the effects of two types of task-agnostic data augmentation on pretrained transformers using 5 classification tasks and 6 datasets. |
| Outcome: | The proposed techniques improve performance on 5 classification tasks, 6 datasets, and 3 variants of modern pretrained transformers. |